Abstract Large vision-language models (LVLMs) have demonstrated their incredible capability in image understanding and response generation. However, this rich visual interaction also makes LVLMs vulnerable to adversarial examples. In this paper, we formulate a novel and practical targeted attack scenario that the adversary can only know the vision encoder of the victim LVLM, without the knowledge of its prompts (which are often proprietary for service providers and not publicly available) and its underlying large language model (LLM). This practical setting poses challenges to the cross-prompt and cross-model transferability of targeted adversarial attack, which aims to confuse the LVLM to output a response that is semantically similar to the attacker’s chosen target text. To this end, we propose an instruction-tuned targeted attack (dubbed I nstruct TA) to deliver the targeted adversarial attack on LVLMs with high transferability. Initially, we utilize a public text-to-image generative model to “reverse” the target response into a target image, and employ GPT-4 to infer a reasonable instruction $$\varvec{p}^\prime$$ p ′ from the target response. We then form a local surrogate model (sharing the same vision encoder with the victim LVLM) to extract instruction-aware features of an adversarial image example and the target image, and minimize the distance between these two features to optimize the adversarial example. To further improve the transferability with instruction tuning, we augment the instruction $$\varvec{p}^\prime$$ p ′ with instructions paraphrased from GPT-4. Extensive experiments on six victim LVLMs demonstrate the superiority of our proposed method in targeted attack performance and transferability. In particular, I nstruct TA achieves an attack success rate of 51.9% on BLIP-2, outperforming the strongest baseline by 10.5%, and consistently yields the highest attack success rates across all evaluated models.

The rapid proliferation of Internet of Things (IoT) devices has revolutionized various industries by enabling smart grids, smart cities, and other applications that rely on seamless connectivity and real‐time data processing. However, this growth has also introduced significant security challenges due to the scale, heterogeneity, and resource constraints of IoT systems. Traditional intrusion detection systems (IDS) often struggle to address these challenges effectively, as they require centralized data collection and processing, which raises concerns about data privacy, communication overhead, and scalability. To address these issues, this paper investigates the application of federated learning for network intrusion detection in IoT environments. We first evaluate a range of machine learning (ML) and deep learning (DL) models, finding that the random forest model achieves the highest classification accuracy. We then propose a federated learning approach that allows distributed IoT devices to collaboratively train ML models without sharing raw data, thereby preserving privacy and reducing communication costs. Experimental results using the UNSW‐NB15 dataset demonstrate that this approach achieves promising outcomes in the IoT context, with minimal performance degradation compared to centralized learning. Our findings highlight the potential of federated learning as an effective, decentralized solution for network intrusion detection in IoT environments, addressing critical challenges, such as data privacy, heterogeneity, and scalability.

In the edge learning process of B5G Internet of Things (IoT) systems, the allocation strategy of edge sensors will directly affect the learning results of the system. This article proposes multiple methods to allocate edge devices that can learn and predict the usage of spectrum data, aiming to improve the efficiency and fairness of edge learning. We design an efficient edge device allocation strategy to enhance the edge learning efficiency and propose a metric called ineffective transmission parameter (ITP) to evaluate its performance. To solve the optimization problem, mathematical analysis is performed and closed-form expressions are obtained. The scenarios considered include: devices with different learning performance and the same learning performance on the same band. We propose three edge device allocation methods: 1) iterative hierarchical Hungarian allocation; 2) bow allocation; and 3) category-divided allocation to ensure the fairness of edge learning among sub-bands. The fairness of the system is measured by evaluating the lowest learning performance in the sub-band. To adapt to the actual scenario, we enhance fairness by introducing band attribute parameters (considering the priority, anti-interference ability, and congestion level of the main users of the sub-band). Simulation results show that the proposed strategy significantly improves the ITP of edge learning in IoT systems, especially in the case of varying band utilization. The fairness scheme improves the overall edge learning fairness of the system.

Recent studies have demonstrated that large language models (LLMs) exhibit exceptional performance across various natural language processing tasks, rivaling or even exceeding human competencies in certain areas [1] – [5] . Typically, LLMs undergo pre-training on extensive text corpora, usually using billions of tokens to develop a foundational model. To better align LLMs with human preferences and directives or to fulfill specific application needs, methods such as supervised fine-tuning (SFT), reinforcement learning from human feedback (RLHF), and direct preference optimization (DPO) have been introduced and demonstrated to be effective. These advancements facilitate more intuitive and efficient human-AI interactions. However, the substantial resource requirements throughout the training process pose challenges for individual users and smaller organizations.

Inferring the fine-grained urban flows based on the coarse-grained flow observations is practically important to many smart city-related applications. However, the collected human/vehicle trajectory flows are usually rather unreliable, may contain various noise and sometimes are incomplete, thus posing great challenges to existing approaches. In this paper, we present a pioneering study on robust fine-grained urban flow inference with noisy and incomplete urban flow observations, and propose a denoising diffusion model named DiffUFlow to effectively address it. Specifically, we propose an improved reverse diffusion strategy. A spatial-temporal feature extraction network called STFormer and a semantic features extraction network called ELFetcher are also proposed. Then, we overlay the spatial-temporal feature map extracted by STFormer onto the coarse-grained flow map, serving as a conditional guidance for the reverse diffusion process. We further integrate the semantic features extracted by ELFetcher to cross-attention layers, enabling the comprehensive consideration of semantic information encompassing the entirety of urban data in fine-grained inference. Extensive experiments on two large real-world datasets validate the effectiveness of our method compared with the state-of-the-art baselines.

The wine industry has a significant socioeconomic impact. Recent advancements in smart agriculture have greatly influenced vine growth and wine-making processes. In this study, we evaluate wine ratings by relating them to multiple wine characteristics. A novel dataset with various features is curated, and an interpretable learning paradigm is proposed. Several different learning models are employed, leading to promising experimental results. Furthermore, through feature contribution analysis, we discover that climate-related factors have become less influential on wine ratings in recent years, highlighting the effectiveness of recent advancements in smart agriculture.

Internet of Things (IoT) is an important infrastructure for supporting vertical applications. However, existing IoT systems are still facing some challenges, e.g., lack of coverage in rural areas and lack of efficient data collection methods. To overcome these challenges, a satellite IoT framework is proposed in this study as a promising solution, and smart agriculture is used as a typical application scenario. A satellite edge computing workflow is further proposed, with computer vision as a case study. In the case study, a lightweight deep learning model named MobileViT is proven effective for aphid detection and infestation severity classification on lemon leaves.

Drought has been a global concern and an effective prediction method is needed. Meteorological data are seen as an efficient and economic approach. Challenges arise with the large volume and high nonlinearity between meteorological variables and the drought level. In this study, deep learning is proposed as an effective solution for drought level prediction as multivariate time series classification. The synthetic minority oversampling technique is further adopted to alleviate the class imbalance problem and improve the classification performance. Experimental results on an open dataset named DroughtED demonstrate the effectiveness of the proposed deep learning method.

With the development of sensing technology, multiple sensors are widely used in Internet of Things (IoT) devices. A key challenge is to learn feature representations from multimodal sensor data to combine the information of different sensors. Although progress has been made by previous works, the correlation between different sensors is still not well exploited, which may limit the performance of representation learning. To address this problem, we propose a deep learning approach to learn representations from multimodal sensor data with maximally correlated autoencoders (MCA). It can efficiently capture high dependence between different modalities at different feature levels. The learned representations are further used for the recognition task. Experimental results on the real-world RGB- D dataset demonstrate the high effectiveness of MCA.

Using content delivery networks (CDNs) for video distribution has become the normalemde factoapproach for video streaming today, because they are easy to use (e.g., video chunks can be delivered as files via HTTP) and have good scalability. Today, it has become the norm rather than the exception for video providers to hire multiple CDNs for their video services in a pay-per-use manner—not only to serve users at different locations, but also to reduce operational costs. Given that multiple CDNs and their peering servers exist at many different locations, selecting different CDNs for different users in the same online video system has become a critical decision that can significantly affect the users’ Quality of Experience (QoE). Conventional strategies are generally rule based, e.g., assigning users to CDNs according to their locations or ISPs, but cannot guarantee any particular QoE level because QoE is affected by a combination of complicated factors. In this article, we propose using a data-driven approach to study the factors determining users’ QoE, including both Quality of Service (QoS) and user factors. Our findings indicate that QoE is affected by both the QoS provided by the CDNs and user preferences for the video content. We design a machine learning-based predictive model to capture the “utility” of a video for a user given a particular QoS guarantee. Based on that, we strategically assign CDNs to users to maximize the overall QoE. One month trace-driven experiments are used to demonstrate the effectiveness and efficiency of our design.

To meet the content delivery requirement of the sky-rocketing increase in video requests, crowdsourced content delivery network (crowdsourced CDN) provides a new promising paradigm for low-cost and low-latency video distribution. However, due to the low contribution of storage and upload bandwidth resources from edge network owners, the resources in crowdsourced CDN are always scare. So how to incentivize crowdsourced resource supply from edge network owners are the key in the crowdsourced CDN paradigm. In this paper, we propose an incentive mechanism to address the challenge. More specifically, a Stackelberg game is formulated to model the âĂIJbargainâĂİ interaction between owners and content provides (CPs). With the game, we propose a genetic algorithm to reach the equibibrium. Finally, trace-driven experiments show that effectiveness of our design.

Compressed bitmap indexes are increasingly used in databases and search engines. By exploiting bit-level parallelism and bitwise operations, e.g. AND/OR operations, they can significantly accelerate the development of many areas. The Word Aligned Hybrid (WAH) bitmap compression scheme using run-length encoding (RLE), is commonly recognized as the most efficient scheme in terms of CPU-performance. This paper presents a new form of compressed bitmap indexes named Adaptive Partitioned Position List Encoding (APPLE), which uses packed position lists for compression. For experiments, we compare it with Huffman encoding, and two enhanced variants of WAH : Concise and COMPAX. Our empirical results show this scheme achieves significant improvement.

Predicting blood glucose concentration facilitates timely preventive measures against health risks induced by abnormal glucose events. Advances in IoT devices, such as continuous blood glucose monitors (CGMs) have made it convenient for measurements of blood glucose in real time. However, accurate and personalized blood glucose concentration prediction is still challenging. Previous inference models yield low-inference accuracy due to the ineffective feature extraction and the limited, imbalanced personal training data. The underlying causal correlations among the blood glucose series are scarcely captured by these models. In this article, we propose CausalBG, a causal recurrent neural network (CausalRNN) deployed on an IoT platform with smartphones and CGM for the accurate and efficient individual blood glucose concentration prediction. CausalBG automatically captures the underlying causal relationships embedded in the blood glucose features through CausalRNN, and efficiently shares the limited personal data among users for the sufficient training via the multitask framework. Evaluations and case studies on 112 users demonstrate that CausalBG significantly outperforms the conventional predictive models on the blood glucose dynamics inference.

In this study we consider the problem of outlier detection with multiple co-evolving time series data. To capture both the temporal dependence and the inter-series relatedness, a multi-task non-parametric model is proposed, which can be extended to data with a broader exponential family distribution by adopting the notion of Bregman divergence. Albeit convex, the learning problem can be hard as the time series accumulate. In this regards, an efficient randomized block coordinate descent (RBCD) algorithm is proposed. The model and the algorithm is tested with a real-world application, involving outlier detection and event analysis in power distribution networks with high resolution multi-stream measurements. It is shown that the incorporation of inter-series relatedness enables the detection of system level events which would otherwise be unobservable with traditional methods.

We propose AutoID, a human identification system that leverages the measurements from existing WiFi-enabled Internet of Things (IoT) devices and produces the identity estimation via a novel sparse representation learning technique. The key idea is to use the unique fine-grained gait patterns of each person revealed from the WiFi Channel State Information (CSI) measurements, technically referred to as shapelet signatures, as the "fingerprint" for human identification. For this purpose, a novel OpenWrt-based IoT platform is designed to collect CSI data from commercial IoT devices. More importantly, we propose a new optimization-based shapelet learning framework for tensors, namely Convex Clustered Concurrent Shapelet Learning (C3SL), which formulates the learning problem as a convex optimization. The global solution of C3SL can be obtained efficiently with a generalized gradient-based algorithm, and the three concurrent regularization terms reveal the inter-dependence and the clustering effect of the CSI tensor data. Extensive experiments are conducted in multiple real-world indoor environments, showing that AutoID achieves an average human identification accuracy of 91% from a group of 20 people. As a combination of novel sensing and learning platform, AutoID attains substantial progress towards a more accurate, cost-effective and sustainable human identification system for pervasive implementations.

Mobility pattern mining answers the fundamental question of where people are likely to go from a given location. It plays an important role in city planning, public transport management and location-based mobile applications. Among these applications, many concern the mobility pattern over contiguous spatial regions as a whole. Traditional ways of mobility pattern mining either result in trip clusters with overlapped origin and destination regions, or require an extra step to partition the city into discrete regions, which may not be optimal for mobility pattern extraction. In this paper, we present a region-aware mobility pattern mining framework to jointly extract trip clusters while maintaining non-overlapping partitions of trip origins and destinations. We developed kernelized ACE, a novel extension to a classic algorithm in statistics to compute the optimal mobility clusters under spatial constraints. Experimental results using Beijing taxi trip data show that our approach outperforms other methods with only ~ 0.3% spatial overlap and 86.43% origin-destination correlation. Our case studies on New York City's and Beijing's taxi datasets also yield insightful findings that reveal city-scale mobility patterns and propose potential improvement for public transportation.

Time series classification is a critical problem in the machine learning field, which spawns numerous research works on it. In this work, we propose AttLSTM-CNNs, an attention-based LSTM network and convolution network that jointly extracts the underlying pattern among the time-series for the classification. The attention-based LSTM automatically captures the long-term temporal dependency among the series, and the CNN describes the spatial sparsity and heterogeneity in the data. The extensive experiments show that the proposed model outperforms the other methods for time-series classification.

Real-time emotion detection has being attracted to human attention recently. Recognizing the inner emotion not only assists people to communicate and understand with each other, but also prevents the occurrence of the serious diseases (e.g., autism) and the emergency (i.e., child abuse, sexual invasion). Existing works usually adopt the professional and cumbersome devices to learn the emotions, and therefore limited in the daily usage. In this work, we design a pervasive and wearable device E-See that enables to recognize the emotion in real time. The prototype of the device is deployed in a microcomputer currently, and it can be resized as a small button worn on the collar or extend as a platform to detect the real-time emotion.

Speech unlocks the huge potentials in emotion recognition. High accurate and real-time understanding of human emotion via speech assists Human-Computer Interaction. Previous works are often limited in either coarse-grained emotion learning tasks or the low precisions on the emotion recognition. To solve these problems, we construct a real-world large-scale corpus composed of 4 common emotions (i.e., anger, happiness, neutral and sadness). We also propose a multi-task attention-based DNN model (i.e., MT-A-DNN) on the emotion learning. MT-A-DNN efficiently learns the high-order dependency and non-linear correlations underlying in the audio data. Extensive experiments show that MT-A-DNN outperforms conventional methods on the emotion recognition. It could take one step further on the real-time acoustic emotion recognition in many smart audio-devices.

Emotion recognition technologies have been widely used in numerous areas including advertising, healthcare and online education. Previous works usually recognize the emotion from either the acoustic or the visual signal, yielding unsatisfied performances and limited applications. To improve the inference capability, we present a multimodal emotion recognition model, EMOdal. Apart from learning the audio and visual data respectively, EMOdal efficiently learns the common and modality-specific information underlying the two kinds of signals, and therefore improves the inference ability. The model has been evaluated on our large-scale emotional data set. The comprehensive evaluations demonstrate that our model outperforms traditional approaches.

Smart buildings today are aimed at providing safe, healthy, comfortable, affordable, and beautiful spaces in a carbon and energy-efficient way. They are emerging as complex cyber-physical systems with humans in the loop. Cost, the need to cope with increasing functional complexity, flexibility, fragmentation of the supply chain, and time-to-market pressure are rendering the traditional heuristic and ad hoc design paradigms inefficient and insufficient for the future. In this paper, we present a platform-based methodology for smart building design. Platform-based design (PBD) promotes the reuse of hardware and software on shared infrastructures, enables rapid prototyping of applications, and involves extensive exploration of the design space to optimize design performance. In this paper, we identify, abstract, and formalize components of smart buildings, and present a design flow that maps high-level specifications of desired building applications to their physical implementations under the PBD framework. A case study on the design of on-demand heating, ventilation, and air conditioning (HVAC) systems is presented to demonstrate the use of PBD.

In the era of Internet of Things, crowd counting, which estimates the number of people within a region, becomes the underpinning for many emerging applications, such as occupancy estimation in smart building and queuing management and product placement in shopping center. Existing vision based crowd counting schemes require favorable lighting conditions and also raise privacy concerns. RF based approaches rely on specialized sensors and require users to carry RF devices. Thus, an accurate, reliable and non-intrusive crowd counting scheme is still desired. In this paper, we propose FreeCount, a device-free crowd counting scheme that is able to precisely estimate the number of people within a region using only commodity WiFi routers. To this end, the channel state information (CSI) data in PHY layer is obtained directly by upgrading the router's software. We propose an information theory based feature selection scheme to select the most representative features that are sensitive to human motion. To build a classifier that is robust to temporal and environmental disparities, we adopt transfer kernel learning, which minimizes the difference between the source and target distributions in the reproducing kernel Hilbert space, is adopted to process the real-time CSI feature data. Experiments were conducted in moderate sized rooms and the results demonstrated that FreeCount is able to accurately estimate the number of people with 96% crowd counting accuracy consistently over temporal and environmental variation.

Human activity recognition is becoming the vital underpinning for a myriad of emerging applications in the field of human-computer interaction, mobile computing, and smart grid. Besides the utilization of up-to-date sensing techniques, modern activity recognition systems also require a machine learning (ML) algorithm that leverages the sensory data for identification purposes. In view of the unique characteristics of the measurement data and the ML challenges thereof, we propose a non-intrusive human activity recognition system that only uses existing commodity WiFi routers. The core of our system is a novel multiple kernel representation learning (MKRL) framework that automatically extracts and combines informative patterns from the Channel State Information (CSI) measurements. The MKRL firstly learns a kernel string representation from time, frequency, wavelet, and shape domains with an efficient greedy algorithm. Then it performs information fusion from diverse perspectives based on multi-view kernel learning. Moreover, different stages of MKRL can be seamlessly integrated into a multiple kernel learning framework to build up a robust and comprehensive activity classifier. Extensive experiments are conducted in typical indoor environments and the experimental results demonstrate that the proposed system outperforms existing methods and achieves a 98\% activity recognition accuracy.

Blood glucose concentration plays an important role in personal health. Hyperglycemia results in diabetes, leading to health risks such as pancreatic function failure, immunity reduce and ocular fundus diseases [6]. Meanwhile, hypoglycemia also brings complications such as confusion, shakiness, anxiety, and if not treated in time, coma or death [2]. People with diabetes need tight control of their blood glucose concentration to avoid both short-term and long-term physiological complications. In this work, we design BGMonitor, the first personalized smartphone-based non-invasive blood glucose monitoring system that detects abnormal blood glucose events by jointly tracking meal, drugs and insulin intake, physical activity and sleep quality. When BGMonitor detects an abnormal blood glucose event, it reminds the user to double-check by finger pricking or using clinical CGM devices.

Existing human activity recognition approaches require either the deployment of extra infrastructure or the cooperation of occupants to carry dedicated devices, which are expensive, intrusive and inconvenient for pervasive implementation. In this paper, we propose SmartSense, a device-free human activity recognition system based on a novel machine learning algorithm with existing commercial off-the-shelf (COTS) WiFi routers. By exploiting the prevalence of existing WiFi infrastructure in buildings, we developed a novel OpenWrt based firmware for COTS WiFi routers to collect the CSI measurements from regular data frames. To identify different human activities, an automatic kernel representation learning method, namely auto-HSRL, is established to selection informative Hilbert space patterns from time, frequency, wavelet, and shape domains. A new information fusion tool based on multi-view kernel learning is proposed to combine the representations extracted from diverse perspectives and build up a robust and comprehensive activity classifier. Extensive experiments were conducted in an office and the experimental results demonstrate that SmartSense outperforms existing methods and achieves a 98% activity recognition accuracy.

Detecting dangerous riding behaviors is of great importance to improve bicycling safety. Existing bike safety precautionary measures rely on dedicated infrastructures that incur high installation costs. In this work, we propose BikeMate, a ubiquitous bicycling behavior monitoring system with smartphones. BikeMate invokes smartphone sensors to infer dangerous riding behaviors including lane weaving, standing pedalling and wrong-way riding. For easy adoption, BikeMate leverages transfer learning to reduce the overhead of training models for different users, and applies crowdsourcing to infer legal riding directions without prior knowledge. Experiments with 12 participants show that BikeMate achieves an overall accuracy of 86.8% for lane weaving and standing pedalling detection, and yields a detection accuracy of 90% for wrong-way riding using crowdsourced GPS traces.

Predicting blood glucose dynamics is vital for people to take preventive measures in time against health risks. Previous efforts adopt handcrafted features and design prediction models for each person, which result in low accuracy due to ineffective feature representation and the limited training data. This work proposes MT-LSTM, a multi-time-series deep LSTM model for accurate and efficient blood glucose concentration prediction. MT-LSTM automatically learns feature representations and temporal dependencies of blood glucose dynamics by jointly sharing data among multiple users and utilizes an individual learning layer for personalized prediction. Evaluations on 112 users demonstrate that MT-LSTM significant outperform conventional predictive regression models.

Metro has become the first choice of traveling for tourists and citizens in metropolis due to its efficiency and convenience. Yet passengers have to rely on metro broadcasts to know their locations because popular localization services (e.g. GPS and wireless localization technologies) are often inaccessible underground. To this end, we propose MetroEye, an intelligent smartphone-based tracking system for metro passengers underground. MetroEye leverages low-power sensors embedded in modern smartphones to record ambient contextual features, and infers the state of passengers (Stop, Running, and Interchange) during an entire metro trip using a Conditional Random Field (CRF) model. MetroEye further provides arrival alarm services based on individual passenger state, and aggregates crowdsourced interchange durations to guide passengers for intelligent metro trip planning. Experimental results within 6 months across over 14 subway trains in 3 major cities demonstrate that MetroEye yields an overall accuracy of 80.5% outperforming the state-of-the-art.

Sleep quality plays a vital role in personal health. A great deal of effort has been paid to design sleep quality monitoring systems, providing services ranging from bedtime monitoring to sleep activity detection. However, as sleep quality is closely related to the distribution of sleep duration over different sleep stages, neither the bedtime nor the intensity of sleep activities is able to reflect sleep quality precisely. We present Sleep Hunter, a mobile service that provides a fine-grained detection of sleep stage transition for sleep quality monitoring and intelligent wake-up call. The rationale is that each sleep stage is accompanied by specific body movements and acoustic signals. Leveraging the built-in sensors on smartphones, Sleep Hunter integrates these physical activities with sleep environment, inherent temporal relation, and personal factors by a statistical model for a fine-grained sleep stage detection. Based on the duration of each sleep stage, Sleep Hunter further provides sleep quality report and smart call service for users. Experimental results from over 30 sets of nocturnal sleep data show that our system is superior to existing actigraphy-based sleep quality monitoring systems, and achieves satisfying detection accuracy compared with dedicated polysomnography-based devices.

Sleep quality plays a significant role in personal health. A great deal of effort has been paid to design sleep quality monitoring systems, providing services ranging from bedtime monitoring to sleep activity detection. However, as sleep quality is closely related to the distribution of sleep duration over different sleep stages, neither the bedtime nor the intensity of sleep activities is able to reflect sleep quality precisely. To this end, we present Sleep Hunter, a mobile service that provides a fine-grained detection of sleep stage transition for sleep quality monitoring and intelligent wake-up call. The rationale is that each sleep stage is accompanied by specific yet distinguishable body movements and acoustic signals. Leveraging the built-in sensors on smartphones, Sleep Hunter integrates these physical activities with sleep environment, inherent temporal relation and personal factors by a statistical model for a fine-grained sleep stage detection. Based on the duration of each sleep stage, Sleep Hunter further provides sleep quality report and smart call service for users. Experimental results from over 30 sets of nocturnal sleep data show that our system is superior to existing actigraphy-based sleep quality monitoring systems, and achieves satisfying detection accuracy compared with dedicated polysomnography-based devices.

Near field authentication is of great importance for a range of applications, and has attracted many research efforts in the past decades. Several approaches have been developed and demonstrated their feasibility. The state-of-art works, however, still have much room to improve their automation and usability. First, user assistance is required in most existing approaches, which will be easily observed and imitated by attackers. Second, the authentications of several works heavily depend on special hardware, e.g., Server or high resolution screen, which greatly restricts their application scenarios. In this paper, we present a near field authentication system Tooth that needs little human assistance and is compatible with most smart phones. ToAuth is based on the key insight that the acceleration traces are similar for a pair of smart phones when they are contacting physically and vibrating. The random vibration patterns are sufficiently uncertain to provide high entropy to generate a pair of cryptographic keys yet are inimitable for a third party who does not get in touch with the vibration source. ToAuth leverages the keys to make authentication for smart phones. We implement ToAuth on Android platform and evaluate its performance under various scenarios. Extensive experiments demonstrate ToAuth could achieve around 90% success rate in stable environment, and prevent attacks depended on vibration noise.

Context-awareness is getting increasingly important for a range of mobile and pervasive applications on nowadays smartphones. Whereas human-centric contexts (e.g., indoor/ outdoor, at home/in office, driving/walking) have been extensively researched, few attempts have studied from phones' perspective (e.g., on table/sofa, in pocket/bag/hand). We refer to such immediate surroundings as micro-environment, usually several to a dozen of centimeters, around a phone. In this study, we design and implement Sherlock, a micro-environment sensing platform that automatically records sensor hints and characterizes the micro-environment of smartphones. The platform runs as a daemon process on a smartphone and provides finer-grained environment information to upper layer applications via programming interfaces. Sherlock is a unified framework covering the major cases of phone usage, placement, attitude, and interaction in practical uses with complicated user habits. As a long-term running middleware, Sherlock considers both energy consumption and user friendship. We prototype Sherlock on Android OS and systematically evaluate its performance with data collected on fifteen scenarios during three weeks. The preliminary results show that Sherlock achieves low energy cost, rapid system deployment, and competitive sensing accuracy.