Provenance-based threat hunting identifies Advanced Persistent Threats (APTs) on endpoints by correlating attack patterns described in Cyber Threat Intelligence (CTI) with provenance graphs derived from system audit logs. A fundamental challenge in this paradigm lies in the modality gap —the structural and semantic disconnect between provenance graphs and CTI reports. Prior work addresses this by framing threat hunting as a graph matching task: 1) extracting attack graphs from CTI reports, and 2) aligning them with provenance graphs. However, this pipeline incurs severe information loss during graph extraction and demands intensive manual curation, undermining scalability and effectiveness.

Abstract In recent years, Living off the Land (LotL) attacks have been drawing attention due to their flexibility and difficulty in detection. These attacks exploit legitimate tools already in the system to conduct malicious activities, hiding their malicious intent behind normal benign programs. However, detection methods for such attacks largely rely on expert rules. While rule tags can effectively detect known attacks, this also leads to a high false positive rate, resulting in low detection accuracy for the models. To address these issues, we propose a detection system called LOTLDetector, which combines deep learning methods with expert rules to detect malicious command lines in LotL attacks from both data and knowledge perspectives. LOTLDetector learns the semantics of command line text through neural networks and combines rule tags from expert knowledge, enabling a more comprehensive detection of LotL attacks. We extensively evaluated our method, validated it on a Windows dataset containing 27,448 command lines and a Linux dataset containing 27,093 command lines, and compared it with existing methods. The results show that our method significantly outperforms existing methods in detecting malicious command lines. For the Linux dataset, the detection system achieved a detection performance with an accuracy of 0.9728; for the Windows dataset, the system’s detection accuracy also reached 0.9598, which is about 8% higher than the best existing method. In addition, our project has been open-sourced at https://github.com/csedikaf/LOTLDetector .

Advanced Persistent Threats (APTs) commonly manifest through a sequence of attack steps, known as Tactics, Techniques, and Procedures (TTPs). Recent studies identify TTPs by converting audit logs into causal provenance graphs and applying expert-driven mappings that correlate low-level system events with high-level TTP patterns. However, these methods face persistent challenges: determining the impact boundaries of TTP activities, adapting to evolving TTP stacks, and recognizing fine-grained TTP semantics for deeper forensic insights. To address these challenges, we presentZoomer, a novel TTP recognition framework that segments provenance graphs into multiple TTP subgraphs with multi-granular annotations (i.e., tactics, techniques, and sub-techniques). First, we devise a heuristic subgraph sampling algorithm guided by anomalous node detection to precisely delineate the scope of TTP activities. Second, we introduce a dual-tower Deep & Wide architecture that integrates contextual behavior semantics from provenance graphs and domain-informed features to learn expressive TTP representations. Finally, we adopt a prototypical network that reformulates TTP recognition as a few-shot pattern matching task, thereby enhancing adaptability and accuracy under limited supervision. To advance future research, we built and released the first TTP-annotated provenance dataset, encompassing the most comprehensive collection of TTP instances to date. Extensive experiments show thatZoomerachieves TTP recognition with 88% accuracy at the sub-technique level and 94% at the tactic level, significantly outperforming state-of-the-art baselines.

As Advanced Persistent Threats (APTs) become more complex and destructive, attack investigation has gained importance. Analysts use provenance graphs for causality analysis on Point-Of-Interest (POI) events to capture critical events. However, existing methods suffer from problems of high false positives, high overhead, and high latency due to the vast size of the provenance graph and the rarity of critical events. We proposeSPARSEfor constructing critical component graphs (i.e., consisting of critical events) from streaming logs in real time. Our approach is based on two key observations: 1) Critical events exist in suspicious semantic graphs (SSGs) composed of interaction flows between suspicious entities, and 2) Information flows accomplishing the attacker's goal exist as paths.SPARSEuses a two-stage framework that first constructs the SSG using a state-based mode with semantic transfer rules and storage strategies. Then, it identifies suspicious flow paths (SFPs) related to the POI event and quantifies each path's influence to filter irrelevant events. Evaluation on a large-scale attack dataset shows our system generates a critical component graph ($\sim$113 edges) in 1.6 seconds, which is 2014 × smaller than the backtracking graph ($\sim$227,589 edges). It is also 25 × more effective in filtering irrelevant edges compared to other state-of-the-art techniques.

With the widespread adoption of microservice architectures, the security threats they face have become increasingly sophisticated. Existing anomaly detection methods based on system calls exhibit significant limitations in three key aspects: multidimensional data fusion, temporal causality modeling, and forensic analysis of anomalies. This paper proposes MADGuard, a provenance graph-based anomaly detection system for microservices. MADGuard addresses these challenges through three key innovations: (1) It constructs a native provenance graph by integrating multisource services and multidimensional data, employing feature hashing and positional encoding for efficient graph representation; (2) The system introduces a Temporal Graph Network (TGN) model combined with edge reconstruction error and Inverse Document Frequency (IDF) weighting, achieving a 15. 07% improvement in the F1 score compared to existing methods; (3) For the first time in microservice security, an integrated forensic analysis module is implemented, allowing rapid anomaly path reconstruction through aggregated anomaly subgraphs. Comprehensive evaluations on typical microservice benchmarks (TeaStore, RobotShop, SockShop) demonstrate MADGuard’s superior performance: 94.08% detection accuracy, significantly outperforming state-of-the-art approaches while maintaining practical operational efficiency.

Network intrusion detection (NID) plays a crucial role in cybersecurity by identifying network attacks from network traffic. In recent years, the deep learning technique has become a tendency for the NID problem. However, a major drawback of deep learning is the lack of interpretability, making NID systems (NIDSs) difficult to diagnose and response to the detected network attacks. At the same time, the existing interpretable deep learning techniques cannot adapt to the NID problem due to its specific challenges, including the cross‐feature effect and the absence of self‐interpretable features. To this end, this article proposes a decision Tree enhanced deep Attention Network (TAN), an interpretable deep learning model specifically designed for the NID problem by integrating a decision tree (DT) into a deep attention network. TAN utilizes a DT to extract self‐interpretable features and then uses a deep hierarchical attention network to capture the cross‐feature effect and pinpoint the most important self‐interpretable features. A series of experiments and case studies were performed on public datasets, including KDD99, NSL‐KDD, UNSW‐NB15, and CICIDS2017. The results indicate that TAN achieves competitive detection performance compared to existing deep learning models, while offering a more intuitive interpretation.

HID (host intrusion detection) is a security mechanism for detecting malicious activities performed in a host (e.g., a server, an edge device). Recent research has recast HID as a provenance graph learning problem thanks to the advancement in deep learning techniques, especially the GNNs (graph neural networks). Although the provenance graph learning based HID methods show promise, they are vulnerable to adversarial attacks, where the attackers can bypass the HID models by carefully modifying their attack behaviors. In this paper, we reveal that an adversarial sample generated against one HID model may not be necessarily able to attack another HID model, and we further explore the success rate of adversarial attacks between different HID models by evaluating the mutual transferability. Based on the evaluation, we propose ProvADShield, a framework designed to defend against adversarial attacks on provenance graph learning based HID models. The core idea of ProvADShield is to combine multiple HID models by leveraging the mutual transferability. We evaluate ProvADShield based on a provenance dataset collected and made public by our team. The experiment results show that ProvADShield outperforms state‐of‐the‐art defense systems against adversarial attacks.

Open source software has evolved into a fundamental element of the contemporary information sector; however, security threats within its supply chain are persistently rising. Within the collaborative development framework of open source, the introduction of malicious code can lead to significant security vulnerabilities. Conventional methods for detecting these vulnerabilities, which rely on machine learning, face challenges such as a lack of sufficient datasets, inadequate deep semantic understanding, and limitations to single-vulnerability detection. To address these challenges, we introduce a novel approach named VulnTrace, which analyzes historical records of submissions in open source projects to construct a high-quality dataset of vulnerabilities with accurate labels. VulnTrace employs Word2Vec alongside Abstract Syntax Tree (AST) technologies to capture both the semantic and structural details of code segments and utilizes a Transformer model for precise vulnerability identification, thereby enhancing accuracy and interpretability in detection. Experimental results indicate that VulnTrace achieves approximately 93% accuracy, 95% precision, 83% recall and an F1 score of 88% in vulnerability detection tasks, significantly reducing false positives and demonstrating remarkable robustness.

Mobile device user authentication technologies have been studied for decades in the context of personal information security. To strike a balance between security, privacy, and usability, authentication methods based on motion sensors have gained widespread attention in recent years. However, these methods still face several challenges, such as the limited training samples, the finite scene coverage, and the high-cost models. Therefore, there is an urgent need to develop more efficient and reliable solutions to enhance the user experience. To address these challenges, we introduce, which offers the following features: 1) It uses a novel data augmentation method (AUTHGANS) to expand the dataset. 2) It employs a differential attention mechanism to reduce noise interference, improve model scene coverage, and simultaneously reduce the model size during the model training phase, and improve the model’s accuracy. 3) It uses a model distillation strategy (AuthFusion), ensuring high accuracy while reducing the model’s computational requirements on devices. Experiments on a dataset with 1,513 users and noise show that achieves high accuracy while requiring less computational power than other state-of-the-art authentication methods.

Cyber Threat Intelligence (CTI) reports are factual records compiled by security analysts through their observations of threat events or their own practical experience with attacks. In order to utilize CTI reports for attack detection, existing methods have attempted to map the content of reports onto system-level attack provenance graphs to clearly depict attack procedures. However, existing studies on constructing graphs from CTI reports suffer from problems such as weak Natural Language Processing (NLP) capabilities, discrete and fragmented graphs, and insufficient attack semantic representation. Therefore, we propose a system called CRUcialG for the automated reconstruction of Attack Scenario Graphs (ASGs) by CTI reports. First, we use NLP models to extract systematic attack knowledge from CTI reports to form preliminary ASGs. Then, we propose a four-phase attack rationality validation framework from the tactical phase with attack procedure to evaluate the reasonability of ASGs. Finally, we implement the relation repair and phase supplement of ASGs by adopting a serialized graph generation model. We collect a total of 10,607 CTI reports and generate 5,761 complete ASGs. Experimental results on CTI reports from 30 security vendors and DARPA show that the similarity of ASG reconstruction by CRUcialG can reach 84.54%. Compared with SOTA (EXTRACTOR and AttackG), the recall of CRUcialG (extraction of real attack events) can reach 88.13% and 94.46% respectively, which is 40% higher than SOTA on average. The F1-score of attack phase validation is able to reach 90.04%.

Advanced Persistent Threat (APT) attacks have caused significant damage worldwide. Various Endpoint Detection and Response (EDR) systems are deployed by enterprises to fight against potential threats. However, EDR suffers from high false positives. In order not to affect normal operations, analysts need to investigate and filter detection results before taking countermeasures, in which heavy manual labor and alarm fatigue cause analysts miss optimal response time, thereby leading to information leakage and destruction. Therefore, we propose Endpoint Forecasting and Interpreting (EFI), a real-time attack forecast and interpretation system, which can automatically predict next move during post-exploitation and explain it in technique-level, then dispatch strategies to EDR for advance reinforcement. First, we use Cyber Threat Intelligence (CTI) reports to extract the attack scene graph (ASG) that can be mapped to low-level system logs to strengthen attack samples. Second, we build a serialized graph forecast model, which is combined with the attack provenance graph (APG) provided by EDR to generate an attack forecast graph (AFG) to predict the next move. Finally, we utilize the attack template graph (ATG) andgraph alignment plus algorithmfor technique-level interpretation to automatically dispatch strategies for EDR to reinforce system in advance. EFI can avoid the impact of existing EDR false positives, and can reduce the attack surface of system without affecting the normal operations. We collect a total of 3,484 CTI reports, generate 1,429 ASGs, label 8,000 sentences, tag 10,451 entities, and construct 256 ATGs. Experimental results on both DARPA Engagement and large scale CTI dataset show that the alignment score between the AFG predicted by EFI and the real attack graph is able to exceed 0.8, the forecast and interpretation precision of EFI can reach 91.8%.

Detecting advanced persistent threats (APTs) at a host via data provenance has emerged as a valuable yet challenging task. Compared with attack rule matching, machine learning approaches offer new perspectives for efficiently detecting attacks by leveraging their inherent ability to autonomously learn from data and adapt to dynamic environments. However, the scarcity of APT samples poses a significant limitation, rendering supervised learning methods that have demonstrated remarkable capabilities in other domains (e.g., malware detection) impractical. Therefore, we propose a system called TAGAPT, which is able to automatically generate numerous APT samples with provenance-level granularity. First, we introduce a deep graph generation model to generalize various graph structures that represent new attack patterns. Second, we propose an attack stage division algorithm to divide each generated graph structure into stage subgraphs. Finally, we design a genetic algorithm to find the optimal attack technique explanation for each subgraph and obtain fully instantiated APT samples. Experimental results demonstrate that TAGAPT can learn from existing attack patterns and generalize to novel attack patterns. Furthermore, the generated APT samples 1) exhibit the ability to help with efficient threat hunting and 2) provide additional assistance to the state-of-the-art (SOTA) attack detection system (Kairos) by filtering out 73% of the observed false positives. We have open-sourced the code and the generated samples to support the development of the security community.

As the scope and impact of cyber threats have expanded, analysts utilize provenance graphs constructed from kernel logs to hunt for threats and investigate attacks. The high frequency of kernel events and the persistence of attacks pose challenges for the efficient storage of provenance graphs. Current approaches can be categorized into two types: pruning-based storage (e.g., LogGC, CPR, and NodeMerge) and encoding-based storage (e.g., DeepZip, SLEUTH, ELISE, and Leonard). However, none of these methods simultaneously satisfy the following three requirements: 1) lossless content, 2) storage efficiency, and 3) query support. To address this gap, we proposeDehydrator, an efficient provenance graph storage system that fulfills all these requirements. For the logs generated by auditing frameworks,Dehydratoruses field mapping encoding to filter field-level redundancy, hierarchical encoding to filter structure-level redundancy, and finally learns a deep neural network to support batch querying. We have conducted evaluations on seven datasets totaling over one billion log entries. Experimental results show thatDehydratorreduces the storage space by 84.55%.Dehydratoris$7.36\times $more efficient than PostgreSQL,$7.16\times $than Neo4j, and$16.17\times $than Leonard (the work most closely related toDehydrator, published at Usenix Security’23).

APT (Advanced Persistent Threat) with the characteristics of persistence, stealth, and diversity is one of the greatest threats against cyber-infrastructure. As a countermeasure, existing studies leverage provenance graphs to capture the complex relations between system entities in a host for effective APT detection. In addition to detecting single attack events as most existing work does, understanding the tactics / techniques (e.g., Kill-Chain, ATT&CK) applied to organize and accomplish the APT attack campaign is also important for security operations. Existing studies try to manually design a set of rules to map low-level system events to high-level APT tactics / techniques. However, the rule based methods are coarse-grained and lack generalization ability. Thus, they can only recognize APT tactics and have difficulty in identifying APT techniques. They also cannot adapt to mutant behaviors of existing APT tactics / techniques.

Abstract Smart contracts have significant losses due to various types of vulnerabilities. However, traditional vulnerability detection methods rely extensively on expert rules, resulting in low detection accuracy and poor adaptability to novel attacks. To address these problems, in this paper, deep learning methods are combined with smart contract vulnerability code detection approaches. Abstract syntax trees (ASTs), which are special isomorphic graph structures, are an important bridge between source code and graph neural networks. By learning the AST, the model can understand the semantics of the source code. Moreover, graph neural networks have an increasing ability to address complex heterogeneous graphs. Therefore, control flow graphs are fused with data flow graphs on the basis of the ASTs to build heterogeneous graphs with richer code semantics. Furthermore, multigranularity analysis of the vulnerability detection results is performed, including coarse-grained contract-level vulnerability detection and fine-grained line-level vulnerability detection. Through this multigranularity detection approach, vulnerabilities in contracts can be identified and analysed more comprehensively, providing a richer perspective and more solutions for vulnerability detection. The experimental results show that the proposed multigranularity vulnerability detection method based on heterogeneous graphs (MVD-HG) improves both the accuracy and range of the detected vulnerability types in contract-level vulnerability detection tasks; moreover, in the line-level vulnerability detection task, the MVD-HG model achieves significant results and addresses the shortcomings of existing methods. In addition, based on code generation methods used in related fields, a data enhancement method based on the source code is developed, which effectively expands the experimental dataset to address the reduced credibility of the results due to insufficient amounts of data.

The authentication technology of mobile device users has been studied for decades. To balance security, privacy, and usability, motion sensors-based user authentication methods are widely investigated in recent years. However, existing studies meet the problems such as scarcity of training samples, underutilization of data, poor de-noising ability, insufficient transferability, privacy leakage, and low accuracy. To overcome these difficulties, we propose a system, calledTrapCog, with the following capabilities: 1) In the phase of data collection,TrapCogcan eliminate man-made noise (mislabeling) through differential training based on down-sampling. 2) In the model training stage, the siamese neural network with Long Short-Term Memory (LSTM) as the sub-network is used to achieve sufficient coverage of sample patterns and the transferability of the model. 3) In the phase of real-world authentication, the privacy of the user is tremendously protected through end-side model deployment and local authentication. Experimental results on a dataset composed of 1,513 users with real-world noise show thatTrapCoghas high accuracy and strong transferability, which is much better than state-of-the-art studies.

Vehicular trajectory data can be widely used in applications such as traffic prediction and congestion control. However vehicular trajectory data is voluminous and requires significant storage and processing resources, which contradicts the resources-constraint vehicular networks. Existing compression methods suffer either low compression effects or privacy leakage. A privacy-aware Trajectory Compression scheme based on Fuzzy markovian Prediction (TCFP) is proposed in this paper, which consists of two steps of fuzzy compression. The first-step compression is achieved by converting the raw trajectory data into fuzzy information on the edge vehicle sides. Further compression is performed at edge RSUs through fuzzy multi-order Markovian prediction combined with new-devised fuzzy deviation filtering rules. Extensive experimental evaluation based on real-world data sets demonstrates the proposed TCFP scheme achieves desired QoS performance in terms of compression rate, compression time and information loss.

Advanced Persistent Threat (APT) attacks have caused massive financial loss worldwide. Researchers thereby have proposed a series of solutions to detect APT attacks, such as dynamic/static code analysis, traffic detection, sandbox technology, endpoint detection and response (EDR), etc. However, existing defenses are failed to accurately and effectively defend against the current APT attacks that exhibit strong persistent, stealthy, diverse and dynamic characteristics due to the weak data source integrity, large data processing overhead and poor real-time performance in the process of real-world scenarios. To overcome these difficulties, in this paper we propose APTSHIELD, a stable, efficient and real-time APT detection system for Linux hosts. In the aspect of data collection, audit is selected to stably collect kernel data of the operating system so as to carry out a complete portrait of the attack based on comprehensive analysis and comparison of existing logging tools; In the aspect of data processing, redundant semantics skipping and non-viable node pruning are adopted to reduce the amount of data, so as to reduce the overhead of the detection system; In the aspect of attack detection, an APT attack detection framework based on ATT&CK model is designed to carry out real-time attack response and alarm through the transfer and aggregation of labels. Experimental results on both laboratory and Darpa Engagement show that our system can effectively detect web vulnerability attacks, file-less attacks and remote access trojan attacks, and has a low false positive rate, which adds far more value than the existing frontier work.

Unsupervised user adaptation aligns the feature distributions of the data from training users and the new user, so a well-trained wearable human activity recognition (WHAR) model can be well adapted to the new user. With the development of wearable sensors, multiple wearable sensors based WHAR is gaining more and more attention. In order to address the challenge that the transferabilities of different sensors are different, we propose SALIENCE (unsupervised user adaptation model for multiple wearable sensors based human activity recognition) model. It aligns the data of each sensor separately to achieve local alignment, while uniformly aligning the data of all sensors to ensure global alignment. In addition, an attention mechanism is proposed to focus the activity classifier of SALIENCE on the sensors with strong feature discrimination and well distribution alignment. Experiments are conducted on two public WHAR datasets, and the experimental results show that our model can yield a competitive performance.

Outcome prediction aims to predict the future health condition of patients from Electronic Health Record (EHR) data. Because of the sequential characteristic of EHR data, recurrent neural network (RNN)-based outcome prediction methods have achieved state-of-the-art results. However, the major drawback of RNN-based outcome prediction methods is lack of interpretability, which would lead to trust issues. Aiming at this problem, this paper proposes interpretable outcome prediction model with hierarchical attention (IoHAN), an interpretable outcome prediction model by leveraging attention mechanism. The main novelty of IoHAN is that it can pinpoint the fine-grained influence on the final prediction result of each medical component by decomposing the attention weights hierarchically into hospital visits, medical variables, and interactions between medical variables. We evaluated IoHAN on MIMIC-III, a large real-world EHR data set. The experiment results demonstrate that IoHAN can achieve higher prediction accuracy than state-of-the-art outcome prediction models. In addition, the hierarchical decomposed attention weights can interpret the prediction results in a more natural and understandable way.

Traffic flow prediction is a fundamental part of ITS (Intelligent Transportation System). Since the correlations of traffic data are complicated and are affected by various factors, traffic flow prediction is a challenging task. Existing traffic flow prediction methods generally take limited static factors (e.g., the distance between sensors and road network topological structure) into consideration and model the correlations of the traffic data separately to predict the future traffic. In this paper, we propose AARGNN (Attentive Attributed Recurrent Graph Neural Network), a GNN (graph neural network) based method considering multiple dynamic factors to predict short-term traffic flow. With multi-source urban data (e.g., POI, road network, incident, weather, etc.), AARGNN considers both static factors and dynamic factors (e.g., spatial distance, semantic distance, road characteristic, road situation, and global context) to predict the short-term traffic flow. Specifically, AARGNN constructs an attributed graph and encodes various factors into the attributes. The correlations of the traffic data are modeled by utilizing the GNN combined with LSTM (long short-term memory). In addition, AARGNN specifies the contributions of each factor based on attention mechanism. Experiments on real-world datasets show that the proposed method outperforms all baseline methods.

Trajectory prediction for mobile phone users is a cornerstone component to support many higher-level applications in LBSs (Location-Based Services). Most existing methods are designed based on the assumption that the explicit location information of the trajectories is available (e.g., GPS trajectories). However, collecting such kind of trajectories lays a heavy burden on the mobile phones and incurs privacy concerns. In this paper, we study the problem of trajectory prediction based on cell-id trajectories without explicit location information and propose a deep learning framework (called DeepCTP) to solve this problem. Specifically, we use a multi-graph embedding method to learn the latent spatial correlations between cell towers by exploiting handoff patterns. Then, we design a novel spatial-aware loss function for the encoder-decoder network to generate cell-id trajectory predictions. We conducted extensive experiments on real datasets. The experiment results show that DeepCTP outperforms the state-of-the-art cell-id trajectory prediction methods in terms of prediction error.

Mobile authentication is a fundamental factor in the protection of user’s private resources. In recent years, motion sensor-based biometric authentication has been widely used for privacy-preserving. However, it faces with the problems including low data collection efficiency, insufficient authentication scenario coverage rate, weak de-noising ability, and poor robustness of models, rendering existing methods difficult to meet the security, privacy, and usability requirements jointly in the real-world scenario. To overcome these difficulties, we propose a system calledEspialCog, which is able to 1) collect the sensor data embedded in mobile devices self-adaptively, unobtrusively and efficiently through the evolutionary stable participation game mechanism (ESPGM) with a high scenario coverage rate; 2) minimize noise from collected data by analyzing three types of abnormalities; and 3) authenticate the ownership of mobile devices in real-time by adopting optimized LSTM model with an enhanced stochastic gradient descent (SGD) algorithm. The simulation experiment on 6000 users shows that the efficiency and coverage rates increase dramatically by deploying our ESPGM. Moreover, we conduct experiments on a large-scale real-world noisy dataset with 1513 users and two other small pure real-world datasets. The experimental results show the high accuracy and favorable robustness ofEspialCogin the noisy environment.

The damage caused by Advanced Persistent Threat (APT) attacks to governments and large enterprises is gradually escalating. Once an attack event is detected, forensic analysis will use the dependencies between system audit logs to rapidly locate intrusion points and determine the impact of the attacks. Due to the high persistence of APT attacks, huge amounts of data will be stored to meet the needs of forensic analysis, which not only brings great storage overhead, but also sharply increases the computing costs. To compact data without affecting forensic analysis, several methods have been proposed. However, in real-world scenarios, we meet the problems of weak cross-platform capability, large data processing overhead, and poor real-time performance, rendering existing data compaction methods difficult to meet the usability and universality requirements jointly. To overcome these difficulties, this paper proposes a general, efficient, and real-time data compaction method at the system log level; it does not involve internal analysis of the program or depend on the specific operating system type, and it includes two strategies: 1) data compaction of maintaining global semantics (GS), which determines and deletes redundant events that do not affect global dependencies, and 2) data compaction based on suspicious semantics (SS). Given that the purpose of forensic analysis is to restore the attack chain, SS performs context analysis on the remaining events from GS and further deletes the parts that are not related to the attack. The results of the real-world experiments show that the compaction ratios of our method to system events are as high as$4.36\times $to$13.18\times $and$7.86\times $to$26.99\times $on GS and SS, respectively, which is better than state-of-the-art studies.

Traffic flow prediction plays an important role in ITS (Intelligent Transportation System). This task is challenging due to the complex spatial and temporal correlations (e.g., the constraints of road network and the law of dynamic change with time). Existing work tried to solve this problem by exploiting a variety of spatiotemporal models. However, we observe that more semantic pair-wise correlations among possibly distant roads are also critical for traffic flow prediction. To jointly model the spatial, temporal, semantic correlations with various global features in the road network, this paper proposes T-MGCN (Temporal Multi-Graph Convolutional Network), a deep learning framework for traffic flow prediction. First, we identify several kinds of semantic correlations, and encode the non-Euclidean spatial correlations and heterogeneous semantic correlations among roads into multiple graphs. These correlations are then modeled by a multi-graph convolutional network. Second, a recurrent neural network is utilized to learn dynamic patterns of traffic flow to capture the temporal correlations. Third, a fully connected neural network is utilized to fuse the spatiotemporal correlations with global features. We evaluate T-MGCN on two real-world traffic datasets and observe improvement by approximately 3% to 6% as compared to the state-of-the-art baseline.

Topic models have achieved big success in recent years. To detect topics in a text stream, various online topic models have been proposed in the literature. The limitations of these works include that: 1) most of them run with fixed topic numbers and 2) the overlaps between the topics may enlarge in the evolving process. Hierarchical topic model is a candidate solution to these problems since it can reveal many useful relationships between the topics. These relationships can help to find high quality topics and reduce topic overlaps. In this paper, a knowledge-based semisupervised hierarchical online topic detection framework is proposed. The proposed framework can detect topics in an online hierarchical way. In addition, it has been proven that introducing external knowledge can improve the performance of text mining. Therefore, the knowledge from external knowledge sources and human experts are also integrated in the proposed framework. Experiments are conducted to evaluate the proposed framework with different metrics. The results show that compared with the baseline methods, our framework can achieve better performance with competitive time efficiency.

Summary Impala system is an open source, analytic MPP database for Apache Hadoop. Impala system uses a query execution scheduling scheme that assigns near‐equal bytes retrieval tasks for different hosts to ensure system load balance. However, such “load balance” cannot guarantee a short response time for Impala system, when there are original loads in the system. Traditional query execution scheduling methods require either some assumptions or particular architecture, which cannot be directly used in Impala system. In this paper, we present a query execution scheduling scheme for Impala system. If the query fetches data from a single table, the scheme exploits the maximum flow algorithm. If the query fetches data from multiple tables, the scheme employs a cost‐based algorithm with heuristic pruning rules. In addition, we propose a cost model for Impala system, which considers parallel execution, communication cost, and cluster load. The performance of the proposed scheme is evaluated by the TPC‐DS benchmark, and experimental results show that the scheme can reduce the query response time by 10%‐30%.

Event detection is a central task for distributed sensor systems and detecting forthcoming events in a timely manner is the main way of minimizing their possibly damaging effects. The state-of-the-art methods for event description and detection always rely on using crisp raw sensory data, which requires huge data transmission as well as is time-consuming. However, even a centralized processing manner cannot ensure accurate event decision due to the imprecision and uncertainty of raw sensor readings. In many cases, users do not care about the raw sensory data or the data format used for in-network processing, but instead they are concerned with the semantic event information, such as “how serious is it?” and “where will it occur?” In addition, the main technique employed by the existing solution for detecting problems is collaboration with neighbors, which requires massive data exchange between neighbors that is highly intensive in terms of wireless communication . In this paper, we introduce an energy-efficient, reliable semantic event information extraction framework using fuzzy sets . Linguistic event variables instead of raw sensor data are used for event information transmission and fusion, and fuzzy method-based semantic event information filtering and fusion algorithms are proposed. Extensive evaluations based on both real-life and synthetic data sets demonstrated that our framework only incurs a small communication cost and it returns interpretable event information with guaranteed accuracy.

Semantic human activity (SHA) refers to users' activities performed in their daily lives (e.g., having dinner, shopping, etc.). SHA recognition is a promising issue in wearable and mobile computing. Most existing methods represent a SHA based on a single view, e.g., representing a SHA as a combination of human body actions, representing a SHA as a distribution of latent semantics. Since SHAs are complicated in nature, single views lack the ability of comprehensively profiling SHAs. In this paper, we propose a bi-view semi-supervised learning based method for recognizing SHAs using accelerometers. First, we represent a SHA based on two different views. One view represents a SHA as a distribution of latent activities in an unsupervised manner, and the other view represents a SHA as a set of human crafted features extracted in a hierarchical way. Second, we use a semi-supervised learning framework, which exploits the complementary information provided by the two views, to improve the classification accuracy based on both labeled and unlabeled data. Extensive experiments show that representing SHAs based on bi-views is more effective than representing SHAs based on single views, and our method is able to yield a competitive SHA recognition performance.

Air pollution has adverse effects on humans and ecosystem, and spatially fine-grained air quality information (i.e., the air quality information of every fine-grained area) can help people to avoid unhealthy outdoor activities. However, the number of air quality monitoring stations is usually limited, and thus spatially fine-grained air quality estimation is a challenging task. This paper proposes a method for inferring spatially fine-grained air quality information throughout a city. On one hand, since air quality is affected by multiple factors (e.g., factory waste gases and automobile exhaust fumes), this method employs various data sources, including traffic, road network, point of interests (POIs), and check-ins from social network services, which are related to air quality, to conduct the estimation. On the other hand, since the labeled data are highly limited due to the sparseness of monitoring stations, this method uses an improved ensemble semi-supervised learning (Semi-EP) to establish the relationship between the various data sources and urban air quality. Semi-EP firstly generates multiple classifiers from the original labeled data set and these classifiers are retrained in the iterative co-training process. Then, ensemble pruning technique is used to select the most-diverse subset from these multiple classifiers. This method is evaluated on the real-world dataset of Hangzhou city, China, and the experimental results have demonstrated its advantages over state-of-the-art methods.

A personally semantic place is a space that is frequently visited by an individual user and carries important semantic meanings (e.g. home, work, etc.) to the user. Many location-aware applications could be greatly enhanced by the ability of automatic discovery of personally semantic places. The discovery of a user's personally semantic places involves obtaining the physical locations and semantic meanings of these places. In this paper, we propose approaches to address both of the problems. For the physical place extraction problem, a hierarchical clustering algorithm is proposed to firstly extract visit points from the GPS trajectories, and then clusters these visit points to form physical places. For the semantic place recognition problem, the temporal, spatial and sequential features in which the places have been visited are explored to categorize them into pre-defined types. An extensive set of experiments conducted based on a dataset of real-world GPS trajectories has demonstrated the effectiveness of the proposed approaches.

Road congestion has been one of the major issues in most metropolises, and thus, it is crucial to detect road congestions effectively and efficiently. Traditional solutions require the deployment of dedicated sensors on the roadside or on the vehicles, which suffer from high installation and maintenance costs and limited coverage. In this paper, we propose an alternative solution by exploiting the sensing ability of mobile phones. However, it is challenging to detect road congestions in a daily-living environment using undedicated mobile phones while guaranteeing energy efficiency. The proposed system only depends on the accelerometer and cellular signal, which have been proven to be energy efficient as compared with other built-in sensors (e.g., GPS). It consists of three interactive modules: (a) an accelerometer-based vehicular movement detection module for detecting the periods when the mobile phone user is traveling by vehicle; (b) a map-matching module relying on the cellular signal for determining the traveled road segments; and (c) a road congestion estimation module for inferring the congestion degree of the traveled road segments. We evaluated the proposed system based on real-world datasets, with promising results.

This article presents the design of iReminder, an intuitive location-based reminder that delivers reminding messages based on users' future routes. iReminder is implemented on mobile phones, and it can predict users' future routes by collecting their daily trajectory data. Then it delivers a reminding message via the mobile phone when it senses that the user is going to the task location. A field study was conducted on how iReminder extends its potential to help users perform everyday tasks and compared the method adopted by iReminder with the method used by traditional location-based reminders. The experimental results show that iReminder outperforms traditional location-based reminder because it delivers reminding messages more appropriately. A detailed discussion is also given to investigate the ideal message delivery point, and the discussion results show that a location-based reminding message is more useful and more likely to be accepted by the user if it is triggered by considering his or her future route.

A place is a locale that is frequently visited by an individual user and carries important semantic meanings (e.g. home, work, etc.). Many location-aware applications will be greatly enhanced with the ability of the automatic discovery of personally semantic places. The discovery of a user's personally semantic places involves obtaining the physical locations and semantic meanings of these places. In this paper, we propose approaches to address both of the problems. For the physical place extraction problem, a hierarchical clustering algorithm is proposed to firstly extract visit points from the GPS trajectories, and then these visit points can be clustered to form physical places. For the semantic place recognition problem, Bayesian networks (encoding the temporal patterns in which the places are visited) are used in combination with a customized POI (i.e. place of interest) database (containing the spatial features of the places) to categorize the extracted physical places into pre-defined types. An extensive set of experiments have been conducted to demonstrate the effectiveness of the proposed approaches based on a dataset of real-world GPS trajectories.

This paper presents a system where the personal route of a user is predicted using a probabilistic model built from the historical trajectory data . Route patterns are extracted from personal trajectory data using a novel mining algorithm, Continuous Route Pattern Mining (CRPM), which can tolerate different kinds of disturbance in trajectory data. Furthermore, a client–server architecture is employed which has the dual purpose of guaranteeing the privacy of personal data and greatly reducing the computational load on mobile devices. An evaluation using a corpus of trajectory data from 17 people demonstrates that CRPM can extract longer route patterns than current methods. Moreover, the average correct rate of one step prediction of our system is greater than 71%, and the average Levenshtein distance of continuous route prediction of our system is about 30% shorter than that of the Markov model based method.