As central hubs of the Internet of Everything, smart phones integrate essential functions such as payments, navigation, and IoT connectivity. However, this expanded functionality also heightens security risks. Motion dynamics biometrics, which utilizes motion patterns from user-phone interactions captured via multi-sensor data, has emerged as a promising solution for smartphone security. Offering continuous and unobtrusive protection by analyzing natural user interactions, it still faces challenges in effectively modeling the complex spatio-temporal dynamics between the user and the phone within multi-sensor data. This paper focuses on leveraging graph neural networks (GNNs) to enhance user behavior modeling for smartphone security protection by capturing the relationships within motion sensor data, but it is non-trivial due to the characteristics of complexity, asynchrony, and temporal dependencies of multi motion sensor data. Towards this end, we propose MotionGNN, a multi-motion spatial-temporal graph based behavior modeling framework for user identification and authentication. Specifically, MotionGNN first divides the input multi-motion sensor data into a sequence of segments adaptively by developing a context-aware data segmentation method. Then, MotionGNN constructs fully connected spatio-temporal graphs to model sensor dependencies and temporal dynamics. Finally, windowing graph convolutions are adopted to learn user behavior representations. To evaluate the performance of MotionGNN, we collect a large-scale dataset from real-world scenarios. Extensive experiments demonstrate the state-of-the-art performance of MotionGNN in user identi fication and authentication tasks. We also test MotionGNN for 7 days on smartphones, showing high authentication accuracy with minimal battery and memory usage, making it a reliable solution for smartphone security protection.

Touch behavior biometric has been widely studied for continuous authentication on mobile devices, which provides a more secure authentication in an implicit process. However, the existing touch behavior biometric-based authentication systems suffer from two issues. First, the existing touch behavior representation methods are hard to characterize touch operations under complex usage context. Second, the authentication accuracy of existing authentication models is inclined to degrade over time in a long-term real-life usage scenario due to change in data distribution caused by varying touch behavior. Toward this end, in this article, we develop IncreAuth, an incremental learning-based continuous authentication framework, which allows to provide effective stable authentication performance in the long-term smartphone usage scenario. Specifically, we first propose a novel context-aware feature set to characterize touch behavior patterns in complex usage context. Then, we develop an authentication model GBDTNN, which integrates the advantages of a gradient boosting decision tree model for processing our high-dimensional feature set and neural network model for efficient online updating. A behavior drift-based online updating mechanism is also designed to learn both long-term and short-term touch behavior patterns. To evaluate our framework, we construct a large-scale smartphone usage data set over two months collected from the unconstrained environment. Extensive experiments demonstrate that IncreAuth achieves the state-of-the-art and stable authentication accuracy over time and low system overheads.

Continuous authentication, which provides identity verification using behavioral biometrics in an implicit and transparent manner, has shown potentials for protecting privacy. As the most common way of human-computer interaction, touch behavior pattern of each user has been proven distinctive and widely adopted for continuous authentication. However, most touch based solutions rely on the touchscreen signals obtained from high-level application programming interfaces, which are hard to characterize fine-grained appearance and contour profile of contact fingertips as well as dynamic sliding information in a touch gesture. In this paper, we propose a continuous authentication framework called CT-Auth, which leverages raw capacitive value collected from capacitive touchscreen on smartphone as a descriptor of touch behavior for authentication. Specifically, we first develop a three-dimensional convolution neural network model for capturing intra-gesture spatial-temporal feature and a structure extraction model for capturing structural information between moving fingertips of a touch gesture and touchscreen. A recurrent neural network based model is also applied for capturing temporal patterns among a sequence of touch gestures. To evaluate the effectiveness of our framework, we recruit 100 volunteers over 2 months and collect a large-scale dataset in the unconstrained conditions. Extensive experiments reveal that CT-Auth provides the state-of-the-art authentication accuracy.

This paper presents a novel map matching framework that adopts deep learning techniques to map a sequence of cell tower locations to a trajectory on a road network. Map matching is an essential pre-processing step for many applications, such as traffic optimization and human mobility analysis. However, most recent approaches are based on hidden Markov models (HMMs) or neural networks that are hard to consider high-order location information or heuristics observed from real driving scenarios. In this paper, we develop a deep reinforcement learning based map matching framework for cellular data, named as DMM, which adopts a recurrent neural network (RNN) coupled with a reinforcement learning scheme to identify the most-likely trajectory of roads given a sequence of cell towers. To transform DMM into a practical system, several challenges are addressed by developing a set of techniques, including spatial-aware representation of input cell tower sequences, an encoder-decoder based RNN network for map matching model with variable-length input and output, and a global heuristics-driven reinforcement learning based scheme for optimizing the parameters of the encoder-decoder map matching model. Extensive experiments on a large-scale anonymized cellular dataset reveal that DMM provides high map matching accuracy and fast inference time.

Retrieving similar trajectories aims to search for the trajectories that are close to a query trajectory in spatio-temporal domain from a large trajectory dataset. This is critical for a variety of applications, like transportation planning and mobility analysis. Unlike previous studies that perform similar trajectory retrieval on fine-grained GPS data or single cellular carrier, we investigate the feasibility of finding similar trajectories from cellular data of multiple carriers, which provide more comprehensive coverage of population and space. To handle the issues of spatial bias of cellular data from multiple carriers, coarse spatial granularity, and irregular sparse temporal sampling, we develop a holistic system cellSim . Specifically, to avoid the issue of spatial bias, we first propose a novel map matching approach, which transforms the cell tower sequences from multiple carriers to routes on a unified road map. Then, to address the issue of temporal sparse sampling, we generate multiple routes with different confidences to increase the probability of finding truly similar trajectories. Finally, a new trajectory similarity measure is developed for similar trajectory search by calculating the similarities between the irregularly-sampled trajectories. Extensive experiments on a large-scale cellular dataset from two carriers and real-world 1,701 km query trajectories reveal that cellSim provides state-of-the-art performance for similar trajectory retrieval.

This paper aims to predict a set of apps a user will open on her mobile device in the next time slot. Such an information is essential for many smartphone operations, e.g., app pre-loading and content pre-caching, to improve user experience. However, it is hard to build an explicit model that accurately captures the complex environment context and predicts a set of apps at one time. This paper presents a deep reinforcement learning framework, named as DeepAPP, which learns a model-free predictive neural network from historical app usage data. Meanwhile, an online updating strategy is designed to adapt the predictive network to the time-varying app usage behavior. To transform DeepAPP into a practical deep reinforcement learning system, several challenges are addressed by developing a context representation method for complex contextual environment, a general agent for overcoming data sparsity and a lightweight personalized agent for minimizing the prediction time. Extensive experiments on a large-scale anonymized app usage dataset reveal that DeepAPP provides high accuracy (precision 70.6 percent and recall of 62.4 percent) and reduces the prediction time of the state-of-the-art by 6.58×. A field experiment of 29 participants demonstrates DeepAPP can effectively reduce launch time of apps.

Predicting the next application (app) a user will open is essential for improving the user experience, e.g., app pre-loading and app recommendation. Unlike previous solutions that only predict which app the user will open, this article predicts both the next app and the time to open it. Time prediction is essential to avoid loading the next app too early and consuming unnecessary resources on smartphones. To predict the next app and open time jointly, we model the app usage sequence as a marked temporal point process (MTPP), whose conditional intensity function can capture the probability of a new app usage event. We develop a novel data-driven MTPP-based app prediction system, named ATPP (App Temporal Point Process), which adopts a recurrent neural network architecture to learn the MTPP conditional intensity function for app prediction. ATPP adopts a set of techniques to incorporate the unique features of app prediction in our RNN architecture, including learning the correlated usage behavior of different apps by representation learning, the temporal dependency of app usage by an attention mechanism, and the location-related app usage behavior by feature extraction and fusion layer. We conduct extensive experiments on a large-scale anonymized app usage dataset to verify ATPP’s effectiveness.

Mobile GPU, as a ubiquitous and powerful accelerator, plays an important role in accelerating on-device DNN (Deep Neural Network) inference. The frequent-upgrade and diversity of mobile GPUs require automatic kernel generation to empower fast DNN deployment. However, current generated kernels have poor performance.

With the wide use of smartphones, more private data are collected and saved in the smartphones. This raises higher requirements for secure and effective user authentication scheme. Continuous authentication leverages behavioral biometrics as identity information and shows promising characteristics for user verification in a continuous and passive means. However, most studies require users to operate the smartphones in a specific mobile application or perform user-defined touch operations. This paper studies the continuous authentication on smartphones in the wild, where it is hard to characterize touching behavior accurately due to the complexity of usage context and cross-use of various types of touch gestures. Towards this end, in this paper, we propose a continuous authentication framework using multiple modalities, named as MMAuth, which integrates the heterogeneous information of user identity from multiple modalities (e.g., motion movement pattern, touch dynamics, usage context). A time-extended behavioral feature set (TEB) and a deep learning based one-class classifier (DeSVDD) are developed for performing more accurate authentication. Evaluations are conducted using a novel unconstrained smartphone usage dataset collected from 100 volunteers in real world as well as a public laboratory dataset. Extensive experimental results demonstrate that the state-of-the-art authentication performance of MMAuth in both unconstrained and laboratory environment, and the effectiveness of its two proposed modules (the TEB feature set and the DeSVDD classifier). Additional experiments on system robustness, in terms of usability to different touch gestures, sensitivity to various mobile applications, and scalability to user space, are also provided to examine the applicability of MMAuth.

Predicting the app that a user will open next is essential for improving user experience, e.g., app pre-loading. Unlike previous solutions that only predict next app’s ID, this work also predicts the time to open next app. Time prediction is important to avoid loading the next app too early, consuming too much memory and energy on smartphones. To predict next app’s ID and open time jointly, we model app usage as a marked temporal point process (MTPP), whose conditional intensity function can capture the probability of a new app usage event. We develop a novel data-driven MTPP-based app prediction system, named as ATPP (App Temporal Point Process), which adopts a recurrent neural network architecture to learn the MTPP conditional intensity function for app prediction. ATPP adopts a set of techniques to incorporate the unique features of app prediction in our RNN architecture, including learning the correlated usage behavior of different apps by representation learning, the temporal dependency of app usage events by an attention mechanism, and the location-related app usage behavior by a feature extraction and fusion layer. We conduct extensive experiments on a large-scale anonymized app usage dataset from 443 users over 21 days. The experiment results demonstrate that ATPP outperforms the state-of-the-art app prediction method by 6.0% in the prediction accuracy of next app ID and 2.09× reduction in the prediction error of next app open time. A field experiment of 22 users reveals that ATPP can reduce the app loading time by 78%.

Different variants of the Vehicle Routing Problem (VRP) have been studied for decades. State-of-the-art methods based on local search have been developed for VRPs, while still facing problems of slow running time and poor solution quality in the case of large problem size. To overcome these problems, we first propose a novel deep reinforcement learning (DRL) model, which is composed of an actor, an adaptive critic and a routing simulator. The actor, based on the attention mechanism, is designed to generate routing strategies. The adaptive critic is devised to change the network structure adaptively, in order to accelerate the convergence rate and improve the solution quality during training. The routing simulator is developed to provide graph information and reward with the actor and adaptive cirtic. Then, we combine this DRL model with a local search method to further improve the solution quality. The output of the DRL model can serve as the initial solution for the following local search method, from where the final solution of the VRP is obtained. Tested on three datasets with customer points of 20, 50 and 100 respectively, experimental results demonstrate that the DRL model alone finds better solutions compared to construction algorithms and previous DRL approaches, while enabling a 5- to 40-fold speedup. We also observe that combining the DRL model with various local search methods yields excellent solutions at a superior generation speed, comparing to that of other initial solutions.

City metro network expansion, included in the transportation network design, aims to design new lines based on the existing metro network. Existing methods in the field of transportation network design either (i) can hardly formulate this problem efficiently, (ii) depend on expert guidance to produce solutions, or (iii) appeal to problem-specific heuristics which are difficult to design. To address these limitations, we propose a reinforcement learning based method for the city metro network expansion problem. In this method, we formulate the metro line expansion as a Markov decision process (MDP), which characterizes the problem as a process of sequential station selection. Then, we train an actor-critic model to design the next metro line on the basis of the existing metro network. The actor is an encoder-decoder network with an attention mechanism to generate the parameterized policy which is used to select the stations. The critic estimates the expected cumulative reward to assist the training of the actor by reducing training variance. The proposed method does not require expert guidance during design, since the learning procedure only relies on the reward calculation to tune the policy for better station selection. Also, it avoids the difficulty of heuristics designing by the policy formalizing the station selection. Considering origin-destination (OD) trips and social equity, we expand the current metro network in Xi'an, China, based on the real mobility information of 24,770,715 mobile phone users in the whole city. The results demonstrate the advantages of our method compared with existing approaches.

Map matching for cellular data is to transform a sequence of cell tower locations to a trajectory on a road map. It is an essential processing step for many applications, such as traffic optimization and human mobility analysis. However, most current map matching approaches are based on Hidden Markov Models (HMMs) that have heavy computation overhead to consider high-order cell tower information. This paper presents a fast map matching framework for cellular data, named as DMM, which adopts a recurrent neural network (RNN) to identify the most-likely trajectory of roads given a sequence of cell towers. Once the RNN model is trained, it can process cell tower sequences as making RNN inference, resulting in fast map matching speed. To transform DMM into a practical system, several challenges are addressed by developing a set of techniques, including spatial-aware representation of input cell tower sequences, an encoder-decoder framework for map matching model with variable-length input and output, and a reinforcement learning based model for optimizing the matched outputs. Extensive experiments on a large-scale anonymized cellular dataset reveal that DMM provides high map matching accuracy (precision 80.43% and recall 85.42%) and reduces the average inference time of HMM-based approaches by 46.58×.

This paper aims to predict the apps a user will open on her mobile device next. Such an information is essential for many smartphone operations, e.g., app pre-loading and content pre-caching, to save mobile energy. However, it is hard to build an explicit model that accurately depicts the affecting factors and their affecting mechanism of time-varying app usage behavior. This paper presents a deep reinforcement learning framework, named as DeepAPP, which learns a model-free predictive neural network from historical app usage data. Meanwhile, an online updating strategy is designed to adapt the predictive network to the time-varying app usage behavior. To transform DeepAPP into a practical deep reinforcement learning system, several challenges are addressed by developing a context representation method for complex contextual environment, a general agent for overcoming data sparsity and a lightweight personalized agent for minimizing the prediction time. Extensive experiments on a large-scale anonymized app usage dataset reveal that DeepAPP provides high accuracy (precision 70.6% and recall of 62.4%) and reduces the prediction time of the state-of-the-art by 6.58×. A field experiment of 29 participants also demonstrates DeepAPP can effectively reduce time of loading apps.

SDN has become the wide area network technology, which the academic and industry most concerned about.The limited table sizes of today’s SDN switches has turned to the most prominent short planks in the network design implementation. TCAM based flow table can provide an excellent matching performance while it really costs much. Even the flow table overflow cannot be prevented by a fixed-capacity flow table. In this paper, we design FTS(Flow Table Sharing) mechanism that can improve the performance disaster caused by overflow. We demonstrate that FTS reduces both control messages quantity and RTT time by two orders of magnitude compared to current state-of-the-art OpenFlow table-miss handler.

This paper aims to accelerate the test-time computation of convolutional neural networks (CNNs), especially very deep CNNs [1] that have substantially impacted the computer vision community. Unlike previous methods that are designed for approximating linear filters or linear responses, our method takes the nonlinear units into account. We develop an effective solution to the resulting nonlinear optimization problem without the need of stochastic gradient descent (SGD). More importantly, while previous methods mainly focus on optimizing one or two layers, our nonlinear method enables an asymmetric reconstruction that reduces the rapidly accumulated error when multiple (e.g., ≥ 10) layers are approximated. For the widely used very deep VGG-16 model [1] , our method achieves a whole-model speedup of 4 × with merely a 0.3 percent increase of top-5 error in ImageNet classification. Our 4 × accelerated VGG-16 model also shows a graceful accuracy degradation for object detection when plugged into the Fast R-CNN detector [2] .

People with low vision, Alzheimer's disease, and autism spectrum disorder experience difficulties in perceiving or interpreting facial expression of emotion in their social lives. Though automatic facial expression recognition (FER) methods on 2-D videos have been extensively investigated, their performance was constrained by challenges in head pose and lighting conditions. The shape information in 3-D facial data can reduce or even overcome these challenges. However, high expenses of 3-D cameras prevent their widespread use. Fortunately, 2.5-D facial data from emerging portable RGB-D cameras provide a good balance for this dilemma. In this paper, we propose an automatic emotion annotation solution on 2.5-D facial data collected from RGB-D cameras. The solution consists of a facial landmarking method and a FER method. Specifically, we propose building a deformable partial face model and fit the model to a 2.5-D face for localizing facial landmarks automatically. In FER, a novel action unit (AU) space-based FER method has been proposed. Facial features are extracted using landmarks and further represented as coordinates in the AU space, which are classified into facial expressions. Evaluated on three publicly accessible facial databases, namely EURECOM, FRGC, and Bosphorus databases, the proposed facial landmarking and expression recognition methods have achieved satisfactory results. Possible real-world applications using our algorithms have also been discussed.

This paper aims to accelerate the test-time computation of deep convolutional neural networks (CNNs). Unlike existing methods that are designed for approximating linear filters or linear responses, our method takes the nonlinear units into account. We minimize the reconstruction error of the nonlinear responses, subject to a low-rank constraint which helps to reduce the complexity of filters. We develop an effective solution to this constrained nonlinear optimization problem. An algorithm is also presented for reducing the accumulated error when multiple layers are approximated. A whole-model speedup ratio of 4× is demonstrated on a large network trained for ImageNet, while the top-5 error rate is only increased by 0.9%. Our accelerated model has a comparably fast speed as the “AlexNet” [11], but is 4.7% more accurate.

An appropriate tool to generate real network traffic plays an important role in testing network system. Traditionally, such a tool relies on software solutions that copies data back and forth between different part of memory to capture or replay network traffic. In this paper, we propose an FPGA-centric approach using parallel logic, which can ensure high accuracy of time and high throughput. We first design an FPGA add-on board dealing with the multifarious work like adding content or calculate statistical value. The system is implemented on an own designed off-the-shelf FPGA network add-on card to demonstrate the viability of our assumption. Experiments demonstrate reasonable performance improvement (higher throughput and replay time precision) when compared with software based solutions.