With the help of advanced sensors, virtual reality (VR) apps provide users with an immersive experience, but they also have the potential to collect a wider range of user data compared to traditional web and mobile apps. As a result, increasing numbers of regulations are being introduced globally, emphasizing the need for app developers to provide privacy policies that inform users about data collection, usage, and sharing (CUS) process. Unfortunately, despite the significant efforts made by VR developers to improve app performance, it remains unclear how they ensure their privacy policies comply with regulations and meet user expectations. In this study, we proposeVPVetto automatically vet privacy policy issues for VR apps. We first summarize five vetting criteria based on a study of privacy policies from popular apps: availability, completeness, granularity, minimization, and consistency. We then dissect VR data and entity ontologies and manually generate VR-related CUS sentences to fine-tune privacy policy language models, overcoming performance degradation when handling VR domain-specific sentences. Finally, we construct the largest VR privacy policy dataset to date, namedVRPP, consisting of privacy policies from 11,923 VR apps across 10 mainstream platforms. These policies were crawled in late 2022 and early 2025 to investigate the evolution of the VR ecosystem. Our vetting process examines platform, app category, and longitudinal perspectives, revealing that VR privacy policies have shown severe privacy issues over the past few years, including limited availability, poor quality, coarse granularity, a lack of adaptation to VR-specific traits, and inconsistencies between CUS statements and actual app behaviors.

During the use of Virtual Reality (VR) applications such as gaming, education, and military training, sensitive information may be generated or collected by VR sensors, raising user concerns about potential data leakage. This highlights the critical need for effective user authentication to prevent unauthorized access. Existing authentication methods for VR are often either cumbersome (e.g., entering passwords via handheld controllers), reliant on specialized hardware (e.g., iris recognition), or vulnerable to credential replay attacks. In this study, we propose PipID, a lightweight VR authentication approach that leverages commercial off-the-shelf (COTS) eye trackers integrated into VR headsets. PipID is based on the fact that users' pupillary responses to visual stimuli vary uniquely. Thus, by displaying lights of randomly selected colors (i.e., wavelengths) on the VR screen, PipID can utilize pupil diameter responses to these wavelengths as the basis for authentication. For pupil data collected by precision-limited COTS eye trackers, PipID mitigates the impact of unrelated eye movements (e.g., blinks) and leverages pupillary response differences between the left and right eyes to further enhance the granularity of authentication features. Additionally, the randomized sequence of light colors helps prevent replay attacks. We implemented PipID on a COTS VR headset and tested it with 52 participants. Experimental results show that PipID achieves an accuracy of 98.65% and maintains robust performance under various conditions (e.g., keeping 98% and 91% accuracy after 7 and 14 days respectively).

Trusted Execution Environment (TEE) has become increasingly popular in privacy-protected cloud computing, and its rapid development has led to the availability of various heterogeneous TEE platforms on cloud servers. To facilitate portable TEE applications on heterogeneous TEE platforms, portable languages or intermediate representations (IRs) with platform-dependent TEE runtimes are adopted. However, existing remote attestation solutions for portable TEE applications follow a nested attestation pattern, i.e., attesting only the TEE runtime and relying on the TEE runtime to measure the loaded portable application, leading to potential security issues. On the other hand, directly packing the TEE runtime and the portable application into an enclave for secure attestation undermines the portability of the portable TEE applications.In this paper, we introduce the concept of portable identities to identify portable TEE applications, and propose a layered attestation framework, Latte, achieving both security and portability in attesting portable TEE applications. We provide a prototype implementation of Latte to validate its practicality, with WebAssembly as the portable IR, and Intel SGX and RISC-V Penglai as the exemplar heterogeneous TEEs. The evaluation demonstrates that Latte introduces minimal performance overhead compared with the nested attestation pattern.

Wireless charging is widely used to charge smart devices with limited battery capacity. However, it is susceptible to the identity spoofing attack, where adversaries can impersonate malicious devices as legitimate ones to gain unauthorized access and potentially disrupt the wireless charging system (e.g., resulting in incorrect billing, overheating, or even explosions). Device fingerprinting is a classical method for defending against identity spoofing attacks. However, applying existing schemes in wireless charging scenarios has drawbacks such as inconvenience (e.g., requiring specialized devices or user participation) and ineffectiveness (e.g., vulnerability to spoofing). Thus, we design a novel passive, effective, and robust device fingerprinting scheme called MagID for wireless charging systems. The insight of MagID lies in the fact that during wireless charging, the magnetic signal around a device can reflect inherent hardware differences. These differences can be extracted as unique fingerprints for authentication purposes. MagID leverages a novel scheme, SUPER-ARRAY, to precisely measure magnetic data and generate effective fingerprints for authenticating a device's identity before starting charging progress. Experimental results demonstrate that MagID achieves an accuracy rate of 98.14% across various charging devices. We have also tested its performance under different impact factors and verified its compatibility with various wireless charging pads.

Hierarchical Deterministic Wallet (HDW) and Stealth Address (SA) are widely used in cryptocurrency communities due to their functionality and security. In the preliminary version of this work (ESORICS 2022), we formally define the syntax and security models of Hierarchical Deterministic Wallet supporting Stealth Address (HDWSA), capturing the functionality and security requirements imposed by the practice in cryptocurrency. We propose a concrete HDWSA construction and prove its security in the random oracle model. Note that when applied in blockchain, in practice, signature aggregation could reduce the cost of computation, storage, and communication dramatically. In this full version, we develop HDWSA definition to further support signature aggregation (referred to as HDWSA$^{2}$). In particular, we first formally define HDWSA$^{2}$, which, besides enjoying all the virtues of HDWSA on functionality and security, allows multiple signatures on different messages to be aggregated into one signature. We propose a concrete HDWSA$^{2}$construction and prove its security in the random oracle model. We implement the HDWSA$^{2}$construction and the experimental results show that verification of an aggregate signature is about 13$\boldsymbol{\times }$faster than sequential verification of all the individual signatures. We can reduce the size of signatures in a single block by about 60% after aggregation.

Currently, the security of the control logic of Programmable Logic Controllers (PLCs) is facing a serious threat, significantly impacting industrial production. Consequently, ensuring the security of PLC control logic becomes imperative. Formal verification emerges as a promising methodology for verifing PLC security through behavioral modeling and security testing. However, existing formal verification approaches primarily focus on modeling the PLC source code, overlooking the identification of compile-time errors and real-time runtime logic checks. Therefore, it is essential to apply formal verification to PLC control logic at the binary level. In this study, we introduce VoICS, a system designed to facilitate binary-level formal verification. Using reverse engineering, VoICS automatically parses PLC programs written by various programming languages at the binary level and constructs control flow graphs (CFGs). Furthermore, we use an algorithm combining two model optimization methods (i.e., trim invalid states and unnecessary states compression) to convert the reversed PLC assembly program into nuXmv format model. Lastly, VoICS establishes the corresponding constraints and performs formal verification on the model using nuXmv. The evaluation results demonstrate the capability of VoICS in identifying instances of unreliable control logic within PLC control programs, thus reinforcing the dependability of the industrial automation system.

In panoramic and immersive virtual reality (VR) scenarios, users type on a floating and invisible keyboard, which cannot be observed by external adversaries, creating the illusion that their input is confidential. While recent studies have demonstrated the feasibility of leveraging side-channel information (e.g., vision, Wi-Fi) to eavesdrop on keystrokes in VR, they assume users typically type with fixed gestures, similar to using traditional physical keyboards. However, in real world scenarios, VR creates a 3D immersive environment, allowing users to type from varying orientations. This variation significantly degrades the quality of side-channel information (e.g., occlusion in vision, instability in Wi-Fi channels), leading to ineffective inference. In this study, we propose a multi-modal keystroke eavesdropping attack called WiViLeak, which combines Wi-Fi and vision information to complement each other. To address low-quality side-channel data caused by users’ varying orientations, we develop a theoretical model to explore the relationship between users’ hand movements in physical space (from the vision modality) and fluctuating Wi-Fi signals (from the wireless modality) as users change orientation. Based on this, we design a fully transformer based orientation calibration module to recover users’ vision data, aligning it as if they were facing the camera (i.e., in a front-facing view). Meanwhile, WiViLeak reconstructs Wi-Fi data to correspond to the front-facing view, utilizing the orientation angle derived from vision data. Finally, WiViLeak extracts effective features from reconstructed, high-quality vision and Wi-Fi data to predict keystrokes. We implement a WiViLeak prototype, achieving 89.2% accuracy in eavesdropping keystrokes and 93.6% top-100 password theft accuracy, while also demonstrating robustness across various real world VR scenarios, including payments, chatting, and meetings.

Mobile advertising has been significantly propelled by the advent of in-app programmatic advertising and Real-Time Bidding (RTB) technologies. However, it suffers from ad fraud incidents in ad networks, including click injection, covert background ad activities, and etc. While previous research has predominantly focused on ad fraud localized within individual apps or specific devices, this paper delineates a newly identified form of collusion-based ad fraud, termed ad attribution laundering fraud (ALF).ALFinvolves multiple apps conspiring to obfuscate the true origins of where advertisements are displayed, thereby allowing lower-quality apps to leverage the reputations of ostensibly legitimate ones. To detectALF, we developed the detection tool, AlfScan-X, which efficiently identifies potential collaborative apps among millions in the wild by heuristically prioritizing candidate apps likely to be involved inALFfor prompt analysis. AlfScan-Xmaintains an online APK crawler and an$\textsf {AppID}$database to enhance AlfScan-X’s responsiveness, adaptability, and reduce false negatives. Overcoming challenges of identity extraction from diverse and obfuscated apps, AlfScan-Xutilizes a combination of static and dynamic analysis techniques to cross-verify app identities, pinpointing instances ofALF. Our evaluation of AlfScan-Xon a 200-app ground truth dataset yielded high effectiveness with 92% precision and 92% recall. AlfScan-Xidentified$4,515$unique fraudulent apps and$1,483$fraudulent clusters, revealing significant patterns and implications of fraudulent apps and highlighting reliability issues in both third-party app development frameworks and advertising networks.

Trusted execution environment (TEE) is a revolutionary technology that enables secure remote execution (SRE) of cloud workloads on untrusted server-side computing platforms. Both commercial and academic TEEs have been proposed in the past few years, including Intel's SGX and TDX, AMD's SEV, ARM's CCA, IBM's PEF, and their academic counterparts built atop open-source RISC-V processors, such as Keystone, Sanctum, CURE, and Penglai. While great efforts from both sides have been made in developing a confidential computing ecosystem, the existence of server-side TEEs with drastically different designs and the presence of various known attacks have significantly increased the difficulty of understanding TEE designs and the reasons behind existing attacks.

Virtual reality (VR) apps can harvest a wider range of user data than web/mobile apps running on personal computers or smartphones. Existing law and privacy regulations emphasize that VR developers should inform users of what data are collected/used/shared (CUS) through privacy policies. However, privacy policies in the VR ecosystem are still in their early stages, and many developers fail to write appropriate privacy policies that comply with regulations and meet user expectations. In this paper, we propose VPVet to automatically vet privacy policy compliance issues for VR apps. VPVet first analyzes the availability and completeness of a VR privacy policy and then refines its analysis based on three key criteria: granularity, minimization, and consistency of CUS statements. Our study establishes the first and currently largest VR privacy policy dataset named VRPP, consisting of privacy policies of 11,923 different VR apps from 10 mainstream platforms. Our vetting results reveal severe privacy issues within the VR ecosystem, including the limited availability and poor quality of privacy policies, along with their coarse granularity, lack of adaptation to VR traits and the inconsistency between CUS statements in privacy policies and their actual behaviors. We open-source VPVet system along with our findings at repository https://github.com/kalamoo/PPAudit, aiming to raise awareness within the VR community and pave the way for further research in this field.

In recent years, the growth of mobile advertising has been driven by in-app programmatic advertising and technologies like Real-Time Bidding (RTB). However, this growth has also led to an increase in ad fraud, such as click injection, background ad activity, etc. While existing studies have primarily concentrated on ad fraud within individual apps or devices, this paper introduces a new form of collusion-based ad fraud, named ad attribution laundering fraud (ALF). ALF involves multiple apps collaborating to deceive advertisers by misrepresenting the app where ads are displayed. The collusion-based approach allows lower-quality apps to exploit the reputable identities of seemingly legitimate apps. This deceives advertisers or ad networks into believing that the advertisements they place are reaching potentially valid end-users on the legitimate app. The seemingly legitimate ad events and ad attribution procedures employed by individual apps in such attacks can evade detection by existing tools.

With the widespread use of smartphones and the development of ad networks, mobile in-app targeted ads have become more and more prevalent, leveraging users' geolocation for targeting purposes. This service involves a large amount of user location data, which may not only expose sensitive locations closely associated with the users, but also reveal the users' activities and profiles. Previous studies have utilized various machine learning methods to infer users' activities or predict their future activities based on the location data from location-based social networks (LBSNs). These approaches, however, often require large datasets for training and are also resource-intensive. Unlike active behaviors, such as checking in, where users intentionally record their location, location data are passively recorded by mobile apps in the background, making inferring activities more challenging. Considering the rapid progress in the reasoning abilities of the large language models (LLMs) in recent years, we aim to evaluate user's activity and profile leakage through LLMs with the assistance of map APIs. We conduct the experiment on the location dataset, which is generated according to specified profiles. The results of the experiment show that the LLM can infer users' activities with an accuracy rate scoring up to 96.1 %, and there is also a high probability of predicting the users' profiles, such as the occupation.

Sharing private data is at risk of potential data breaches, including the violation of the “right to be forgot-ten” principle, undermining people's willingness to share their data. A common solution is to involve the Trusted Execution Environment (TEE), which allows the data provider to verify the computation process without trusting others. However, previous works have either encountered incomplete computations or lacked scalability. In this paper, we propose TEERASE,a secure data-sharing framework that addresses these issues. TEERASEprotects every phase of the data lifecycle and enables individuals to share personal data with a predefined privacy budget. In particular, TEERASEapplies comprehensive privacy budgeting mechanisms to efficiently manage privacy budgets and employs an asynchronized execution approach that decouples budget consumption from data computation. TEERASErecords the predefined privacy budgets, verifies privacy consumption requests, updates the remaining budgets, and deletes data that have exhausted their budgets by preventing any attempts to access them. We implement a prototype of TEERASEand evaluate its effectiveness with a realistic case study on Genome-Wide Association Study.

Cross-platform UI frameworks may facilitate a new category of Potentially Unwanted Apps, dubbed XPUAs, which uses framework-specific language to implement its UI in the form of cross-platform payload. XPUAs are able to bypass the existing app vetting procedures leveraging their unique technical characteristics and make revenue on addicitive contents that are strictly prohibited by either local laws or app market regulations. In this paper, we first examined the profit chain of XPUAs and then proposed PUAXray, a novel detection system that utilized machine learning to identify XPUAs. PUAXray used a binary classifier that was trained on features extracted from cross-platform payloads, including semantics information and third-party library usage information. We evaluated PUAXray on a dataset that was created for the first time in the community with benign apps from reputable app markets and XPUAs from an industry collaborator. PUAXray achieved 95.4% F1-score in the XPUAs identification task, and proved capable to be extended to other cross-platform UI frameworks.

Recent works have demonstrated that trusted execution environments, such as Intel Software Guard Extensions, are vulnerable to various attacks from the privileged software, including side-channel attacks. Existing solutions, such as T-SGX, Déjà Vu, Cloak and Varys, detect side-channel attacks at runtime. But they are limited by design, because false detection is unavoidable in these detection methods and therefore any security policy developed atop these mechanisms has to tolerate some malicious operations to achieve practical false positive detection rates. In this article, we propose the concept of verifiable execution contracts, which request the privileged software to provide a benign execution environment for enclaves within which launching attacks becomes very difficult, if not impossible. Since the privileged software is untrusted, we design methods for verifying that the execution contracts are observed. With the proposed verifiable execution contracts, we analyzed how existing attacks could be mitigated.

In this paper, we propose a novel and practical mechanism to enable the service provider to verify whether a suspect model is stolen from the victim model via model extraction attacks. Our key insight is that the profile of a DNN model's decision boundary can be uniquely characterized by its Universal Adversarial Perturbations (UAPs). UAPs belong to a low-dimensional subspace and piracy models' subspaces are more consistent with victim model's subspace compared with non-piracy model. Based on this, we propose a UAP fingerprinting method for DNN models and train an encoder via contrastive learning that takes fingerprints as inputs, outputs a similarity score. Extensive studies show that our framework can detect model Intellectual Property (IP) breaches with confidence > 99.99 % within only 20 fingerprints of the suspect model. It also has good generalizability across different model architectures and is robust against post-modifications on stolen models.

Abstract This paper presents Specularizer , a framework for uncovering speculative execution attacks using performance tracing features available in commodity processors. It is motivated by the practical difficulty of eradicating such vulnerabilities in the design of CPU hardware and operating systems and the principle of defense-in-depth. The key idea of Specularizer is the use of Hardware Performance Counters and Processor Trace to perform lightweight monitoring of production applications and the use of machine learning techniques for identifying the occurrence of the attacks during offline forensics analysis. Different from prior works that use performance counters to detect side-channel attacks, Specularizer monitors triggers of the critical paths of the speculative execution attacks, thus making the detection mechanisms robust to different choices of side channels used in the attacks. To evaluate Specularizer , we model all known types of exception-based and misprediction-based speculative execution attacks and automatically generate thousands of attack variants. Experimental results show that Specularizer yields superior detection accuracy and the online tracing of Specularizer incur reasonable overhead.

Intel Software Guard Extensions (SGX) remote attestation enables enclaves to authenticate hardware inside which they run, and attest the integrity of their enclave memory to the remote party. To enforce direct control of attestation, Intel mandates attestation to be verified by Intel's attestation service. This Intel-centric attestation model, however, neither protects privacy nor performs efficiently when distributed and frequent attestation is required. This paper presents OPERA, an Open Platform for Enclave Remote Attestation. Without involving Intel's attestation service while conducting attestation, OPERA is unchained from Intel, although it relies on Intel to establish a chain of trust whose anchor point is the secret rooted in SGX hardware. OPERA is open, as the implementation of its attestation service is completely open, allowing any enclave developer to run her own OPERA service, and its execution is publicly verifiable and hence trustworthy; OPERA is privacy-preserving, as the attestation service does not learn which enclave is being attested or when the attestation takes place; OPERA is performant, as it does not rely on a single-point-of-verification and also reduces the latency of verification.

Speculative execution side-channel vulnerabilities in micro-architecture processors have raised concerns about the security of Intel SGX. To understand clearly the security impact of this vulnerability against SGX, this paper makes the following studies: First, to demonstrate the feasibility of the attacks, we present SgxPectre Attacks (the SGX-variants of Spectre attacks) that exploit speculative execution side-channel vulnerabilities to subvert the confidentiality of SGX enclaves. We show that when the branch prediction of the enclave code can be influenced by programs outside the enclave, the control flow of the enclave program can be temporarily altered to execute instructions that lead to observable cache-state changes. An adversary observing such changes can learn secrets inside the enclave memory or its internal registers, thus completely defeating the confidentiality guarantee offered by SGX. Second, to determine whether real-world enclave programs are impacted by the attacks, we develop techniques to automate the search of vulnerable code patterns in enclave binaries using symbolic execution. Our study suggests that nearly any enclave program could be vulnerable to SgxPectre Attacks since vulnerable code patterns are available in most SGX runtimes (e.g., Intel SGX SDK, Rust-SGX, and Graphene-SGX). Third, we apply SgxPectre Attacks to steal seal keys and attestation keys from Intel signed quoting enclaves. The seal key can be used to decrypt sealed storage outside the enclaves and forge valid sealed data; the attestation key can be used to forge attestation signatures. For these reasons, SgxPectre Attacks practically defeat SGX's security protection. Finally, we evaluate Intel's existing countermeasures against SgxPectre Attacks and discusses the security implications.

Searchable encryption enables searches to be performed on encrypted documents stored on an untrusted server without exposing the documents or the search terms to the server. Nevertheless, the server typically learns which encrypted documents match the query-the so-called access pattern-since the server must return those documents. Recent studies have demonstrated that access patterns can be used to infer the search terms in some scenarios. In this paper, we propose a framework to protect systems using searchable symmetric encryption from access-pattern leakage. Our technique is based on d-privacy, a generalized version of differential privacy that provides provable security guarantees against adversaries with arbitrary background knowledge.

In this paper, we present HYPERRACE, an LLVM-based tool for instrumenting SGX enclave programs to eradicate all side-channel threats due to Hyper-Threading. HYPERRACE creates a shadow thread for each enclave thread and asks the underlying untrusted operating system to schedule both threads on the same physical core whenever enclave code is invoked, so that Hyper-Threading side channels are closed completely. Without placing additional trust in the operating system's CPU scheduler, HYPERRACE conducts a physical-core co-location test: it first constructs a communication channel between the threads using a shared variable inside the enclave and then measures the communication speed to verify that the communication indeed takes place in the shared L1 data cache-a strong indicator of physical-core co-location. The key novelty of the work is the measurement of communication speed without a trustworthy clock; instead, relative time measurements are taken via contrived data races on the shared variable. It is worth noting that the emphasis of HYPERRACE's defense against Hyper-Threading side channels is because they are open research problems. In fact, HYPERRACE also detects the occurrence of exception-or interrupt-based side channels, the solution.s of which have been studied by several prior works.

Side-channel risks of Intel's SGX have recently attracted great attention. Under the spotlight is the newly discovered page-fault attack, in which an OS-level adversary induces page faults to observe the page-level access patterns of a protected process running in an SGX enclave. With almost all proposed defense focusing on this attack, little is known about whether such efforts indeed raises the bar for the adversary, whether a simple variation of the attack renders all protection ineffective, not to mention an in-depth understanding of other attack surfaces in the SGX system. In the paper, we report the first step toward systematic analyses of side-channel threats that SGX faces, focusing on the risks associated with its memory management. Our research identifies 8 potential attack vectors, ranging from TLB to DRAM modules. More importantly, we highlight the common misunderstandings about SGX memory side channels, demonstrating that high frequent AEXs can be avoided when recovering EdDSA secret key through a new page channel and fine-grained monitoring of enclave programs (at the level of 64B) can be done through combining both cache and cross-enclave DRAM channels. Our findings reveal the gap between the ongoing security research on SGX and its side-channel weaknesses, redefine the side-channel threat model for secure enclaves, and can provoke a discussion on when to use such a system and how to use it securely.

Visual (V) surveillance systems are extensively deployed and becoming the largest source of big data. On the other hand, electronic (E) data also plays an important role in surveillance and its amount increases explosively with the ubiquity of mobile devices. One of the major problems in surveillance is to determine human objects' identities among different surveillance scenes. Traditional way of processing big V and E datasets separately does not serve the purpose well because V data and E data are imperfect alone for information gathering and retrieval. Matching human objects in the two datasets can merge the good of the two for efficient large-scale surveillance. Yet such matching across two heterogeneous big datasets is challenging. In this paper, we propose an efficient set of parallel algorithms, called EV-Matching, to bridge big E and V data. We match E and V data based on their spatiotemporal correlation. The EV-Matching algorithms are implemented on Apache Spark to further accelerate the whole procedure. We conduct extensive experiments on a large synthetic dataset under different settings. Results demonstrate the feasibility and efficiency of our proposed algorithms.

Accurate indoor localization is a key enabling technology for numerous applications such as indoor navigation, mobile social networking, and augmented reality. Despite major effort from the research community, state-of-the-art indoor localization performance remains unsatisfactory. Current approaches using radio frequency entail tedious site surveys and have limited accuracy. While vision-based localization techniques are promising, they struggle with human recognition and changing environments. This paper proposes Flash-Loc, an accurate indoor localization system leveraging flashes of light to localize people carrying mobile phones in areas with deployed surveillance cameras. A person's mobile phone emits a sequence of flashes that uniquely “represents” the person from the cameras' view. Flash-Loc develops three key mechanisms that distinguish people while avoiding long irritating flashes: adaptive-length flash coding, pulse width modulation based flash generation, and image subtraction based flash localization. Further, we design a system in which Flash-Loc cooperates with fingerprinting and dead reckoning for accurate human localization. We implement Flash-Loc on commercial off-the-shelf equipment. Our real-world experiments show Flash-Loc achieves accurate indoor localization by itself and in cooperation with other localization technology. In particular, Flash-Loc can localize a user 45 m away from the camera with sub-meter accuracy.

Smart devices with built-in sensors, computational capabilities, and network connectivity have become increasingly pervasive. Crowds of smart devices offer opportunities to collectively sense and perform computing tasks at an unprecedented scale. This paper presents Crowd-ML, a privacy-preserving machine learning framework for a crowd of smart devices, which can solve a wide range of learning problems for crowd sensing data with differential privacy guarantees. Crowd-ML endows a crowd sensing system with the ability to learn classifiers or predictors online from crowd sensing data privately with minimal computational overhead on devices and servers, suitable for practical large-scale use of the framework. We analyze the performance and scalability of Crowd-ML and implement the system with off-the-shelf smartphones as a proof of concept. We demonstrate the advantages of Crowd-ML with real and simulated experiments under various conditions.