As large-scale systems continue to grow in complexity, effectively leveraging the multi-dimensional information contained in semi-structured logs has become a critical challenge for ensuring system stability. Existing methods rely on limited log content, which restricts semantic modeling capability and leads to degraded performance when abnormal samples are scarce or data distributions are imbalanced. To address these challenges, we propose RELog, an efficient and robust anomaly detection framework that comprehensively utilizes multi-dimensional log features. RELog adopts a hierarchical process for efficient log parsing and anomaly detection. During log parsing, Large Language Models are introduced to enhance the semantic understanding of low-confidence templates generated by the heuristic method. For anomaly detection, we design dedicated encoders for templates, parameters, and time features, tailored to the low complexity and high redundancy of logs, enabling discriminative embeddings. Furthermore, we propose an efficient hybrid sequence encoder integrating state-space modeling and attention for capturing continuous log patterns and critical dependencies, complemented by a replacement classification task for imbalanced data. Experimental results demonstrate that RELog achieves high-accuracy log parsing and anomaly detection while maintaining efficiency. Moreover, RELog demonstrates robustness across varying anomaly ratios and dynamic environments, effectively detecting diverse types of anomalies reflected by parameter and time features.

With continuous growth of IoT applications, service failures are quite inevitable. Due to the complexity and dynamics of IoT services, the root cause analysis (RCA) following an alert can assist in quickly resolving the possible faults. However, the time scales of metrics (e.g., CPU utilization, memory usage) generated by microservices and the dynamic topologies generated by calls between the Application Program Interfaces (APIs) are different. Moreover, the status of devices is an important aspect of RCA in IoT. All these make it extremely challenging to learn failure features of microservice metrics and API calls. Therefore, we propose a novel framework for collaborative identification of root cause analysis (CIRCA) to identify the most potential root cause path with the highest fault scores (weights). In detail, we use both microservice-level and API-level root cause identification (RCI) models to obtain the node fault score in the path. Since we prove the root cause path inference problem is an NP-hard problem, and we then propose a topology-based weighted variable neighborhood search (TWVNS) algorithm and infer the optimal root cause path from two-level scores and call topologies. Our experiments demonstrate CIRCA achieves satisfactory results of RCI and path inference on four public datasets.

Multivariate time series forecasting has a wide range of applications in the fields of computer supported cooperative work. In recent years, deep learning models like transformers, have been widely used in time series forecasting. However, traditional transformers mainly capture short-term dependencies (point to point) within single-variable sequences and struggle with capturing cross-variable and multi-scale dependencies for long-term forecasting. To address this issue, we propose a novel transformer model called MLPT, an MLP-Enhanced Patch Transformer for Multivariate Time Series Forecasting. Our model extracts global features, reconnects them with patches and then calculates both inter and intra attenions, being capable of capturing dependencies in time series from both inter and intra variables, and models these dependencies globally, which makes our model outperform in long-term forecasting. We evaluate our method using seven real-world public datasets, and the experimental results demonstrate that our approach has improved over existing methods in the vast majority of scenarios, with an overall improvement of above 5% on the accuracy. At the same time, the time cost is reduced to one-fourth compared with MTGNN.

With the rapid advancement of information technology, the volume of log data generated by information systems has expanded significantly. The effective detection and processing of anomalies in log sequences has emerged as a crucial research topic. Recent studies have successfully applied pre-trained models for anomaly detection in log sequences. However, due to the semi-structured nature of logs and the presence of domain-specific vocabulary, along with the diverse logging styles of different vendors, those models struggled to provide a comprehensive understanding and unified representation of logs. Additionally, current research primarily focused on log template sequences, neglecting the impact of variable values and contextual relationships among logs. In this paper, we propose LlmBertLog, a BERT-based log sequence anomaly detection model guided by large language models. We first utilize a large language model to transform a limited set of raw logs into natural language descriptions as sample data. Subsequently, we induct three novel pre-training tasks designed to enhance BERT's comprehension and representation capabilities through methods such as semantic augmentation and semantic masking. We then integrate a Transformer with positional encoding and fine-tune LlmBertLog for log sequence anomily detection task, thereby improving its performance in log sequence content mining. Extensive experiments demonstrate that LlmBertLog achieves F1 scores of 0.994 on the BGL dataset and 0.998 on the HDFS dataset for log sequence anomaly detection, surpassing the current best benchmarks by 2.3% and 5.5%, respectively.

Multivariate time series forecasting has a wide range of applications in scenarios such as wireless communication and mobile computing. Recent research has demonstrated that transformer architectures are highly effective for time series forecasting, with smaller models achieving impressive results across multiple public datasets. However, smaller models tend to underperform in scenarios where data is scarce, such as in zero-shot prediction tasks. In recent years, with the rise of large language models, there has been growing interest in developing large models for time series prediction, leading to a series of advancements in this area. However, the current models often suffer from issues such as neglecting the inter-dependencies between multi-dimensional variables and their operations are too complicated. In this paper, focusing on zero-shot prediction, we enhance the Timer model by efficiently and adaptively capturing multivariate dependencies and frequency domain dynamics. Experiments show that the zero-shot prediction capability of our model has significantly improved on multiple public datasets, with an decrease in MSE for over 10%.

Anomaly detection of sensor data is crucial to ensure the stability and effectiveness of Internet of Things (IoT) system. The task requires high accuracy and low latency, which makes distributed anomaly detection gradually become a research hotspot. Edge AI networks further enhance the computing power of edge servers, allowing distributed anomaly detection to be gradually applied to complex scenarios under the Industrial IoT (IIoT), such as smart factories. However, in such cases, the data reported by different sensors in different fields are not independent and identically distributed (non-IID). Simultaneously, during unsupervised training, anomaly data mixed into the training data will reduce the recognition ability of the anomaly detector. To address these challenges, we propose an unsupervised distributed anomaly detection framework. On the cloud, we train an unsupervised anomaly detection model with global factors, using global and local factors to learn the distribution patterns of different fields. At the edge, a multidimensional threshold and its automatic selection algorithm are proposed to overcome the problem of decreased anomaly recognition ability introduced by anomaly training data. Extensive experiments on six datasets show that our approach outperforms SOTA methods in F1-score, can detect anomalies with high accuracy and efficiency in distributed IoT scenarios.

Logs record various operations and events during system running in text format, which is an essential basis for detecting and identifying potential security threats or system failures, and is widely used in system management to ensure security and reliability. Existing log sequence anomaly detection is limited by log parsing and does not consider all key features of logs, which may cause false or missed detection. In this article, we propose a fast and accurate log parsing method and feed the entire log content into the deep learning network for analysis. To avoid semantic loss during parsing, we replace some variables with tokens containing semantic information and divide logs with appropriate granularity. To ensure the speed and accuracy of parsing, we propose a similarity-based fast merging method to deal with redundant templates. For anomaly detection, we use the complete log content features as input to the model. We use Bidirectional Encoder Representation from Transformers (BERT) to output anomaly detection results directly after considering both the global and local information of log sequences. Experiments show that our log parsing method achieves the best average parsing quality on 16 datasets, and the anomaly detection method achieves optimal results on different datasets.

The advent of the cloud-native era, driven by technologies like cloud computing, big data, and virtualization, has promoted the widespread adoption of microservices architecture in application development. The complex network environment and componentized distributed deployments have introduced new challenges in application performance monitoring. This paper introduces TopoRCA, a system that leverages application topology to identify the root causes of issues in microservices applications. TopoRCA utilizes lightweight machine learning models for anomaly detection, locating faults within the system. It also efficiently and intelligently infers the root cause behind system failures by correlating instance node metrics and service call metrics. We use a microservices dataset consisting of 10 service instances in our experiment. Our experimental results demonstrate that 42% of the root causes are ranked first, 61% in the top three, and 80% in the top five by TopoRCA.

Encrypted traffic classification is a challenging task characterized by its invisibility of content, class imbalance, and limited number of labeled samples. In current research, the application of pre-trained language models has achieved significant success. However, employing language models for encrypted traffic classification on resource-constrained devices requires reducing model size and inference cost while preserving accuracy. In this paper, we proposed an unsupervised, task-agnostic, transformer-based language model knowledge distillation method for encrypted traffic data, called NetKD. It leverages Multi-Head Self-Attention Relation Alignment, Masked BURST Model, and Same-origin BURST Prediction tasks, to effectively transfer the byte-level contextual knowledge about encrypted traffic data learned by a large-scale "teacher" BERT model, to a compact "student" NetKD-BERT model. The student model only requires a small amount of supervised data for fine-tuning then can be applied to downstream tasks. Experiments show that compared to the state-of-the-art baseline model, our model uses only 4.61% of the parameters, achieves on average 99.10% of its F1 score in 4 downstream tasks, with a classification speed of 0.93ms/packet. Its memory footprint is as low as 7.33% of the baseline model’s, leading to a significant 14.3 times faster inference speed.

Chinese text Classification models are vulnerable to adversarial attacks. Based on Chinese language features such as phonology and glyphs, attackers can modify serval words or characters and perturb the results of the classification model without affecting the semantics of the sentence. Adversarial examples are becoming serious challenges to the robustness of the classification model, even for the state of the art models such as PLMs. Therefore, it is necessary to effectively improve the robustness of text classification models with Chinese language features. In this paper, we propose an efficient defense method CWordDefender to address the adversarial robustness problem in Chinese text classification tasks. We extract multimodal information based on Chinese features and fine-tune PLMs with contrastive-adversarial learning. Experimental results show that CWordDefender is superior to the baseline model by at least 5% in accuracy and has lower infer time.

With the fast development of Internet of Things (IoT), anomaly detection has recently become a common concern for IoT smart applications. The circuit detection mode of all services to detect application anomaly is widely adopted. However, regularly collecting key performance indicators (KPIs) of all services under different clouds is a time-consuming task. Moreover, too many alerts in a short period of time will affect the processing speed of engineers. Recently, some researches of inferring the key service on the call paths can solve the above problems. However, it is still a challenge to locate dynamic and scattered key services accurately. In this article, we propose a service inference-based anomaly detection approach (SIADA), which integrates three sources of logs: 1) call; 2) business; and 3) metric. SIADA leverages the deep graph representation with context-aware multigraph fusion based on a recurrent encoder. This should infer key services and adopt variational autoencoder (VAE) with the flow model to detect multivariate time-series anomalies for key services. We have conducted extensive experiments on the public data set MicroSS. SIADA achieves the best average accuracy of 92% in service inference and the best average F1-score of 0.98 in anomaly detection, which has improved by 12.17% and 6.42% compared with the best SOTA baseline, respectively. Moreover, the total detection time, network transmission, and average alert number are reduced by 42.12%, 81.87%, and 83.5%, respectively.

In the production environment of the Internet of Things (IoT), sensors of various qualities generate a large amount of multivariate time series (MTS) data. The long-term prediction of time series data generated by various IoT devices provides longer foresight and helps execute necessary resource scheduling or fault alarms in advance, thus improving the efficiency of system operation and ensuring system security. In recent years, deep learning models like Transformers have achieved advanced performance in multivariate long-term time series forecasting (MLTSF) tasks. However, many previous research attempts either overlooked the interseries dependencies or ignored the need to model the strict temporal order of MTS data. In this article, we introduce GTformer, a graph-based temporal-order-aware transformer model. We propose an adaptive graph learning method specifically designed for MTS data to capture both uni-directional and bi-directional relations. In addition, we generate positional encoding in a sequential way to emphasize the strict temporal order of time series. By adopting these two components, our model can have a better understanding of the interseries and intraseries dependencies of MTS data. We conducted extensive experiments on eight real-world data sets, and the results show that our model achieves better predictions compared with state-of-the-art methods.

Mobile edge computing (MEC) emerges as a promising paradigm that aims to extend the computing capabilities of mobile devices. This enhancement is achieved by offloading heavy computational tasks to nearby edge servers, where relevant services, such as programs, databases, and libraries, are stored to support the execution of these tasks. Given the restricted storage capacity and computing resources of edge servers, it is crucial to carefully consider which tasks to offload and which services to cache. However, the highly dynamic system and closely coupled decisions between offloading and caching pose considerable challenges to developing an effective strategy. In this article, we investigate both computation offloading and service caching in an end-edge–cloud collaborative system under time-varying wireless channels and task requests. We formulate an optimization problem aimed at minimizing the system cost, including time and energy consumption, and model it further as a Markov decision process (MDP). To address this, we propose a deep reinforcement learning (DRL)-based approach with dynamic action masking to concurrently optimize offloading and caching decisions. This approach renormalizes the valid action probability distribution at every step, enabling the agent to explore within the space of valid action combinations and thereby learn the optimal policy. Extensive experimental results demonstrate that the proposed approach greatly reduces the system cost compared to other representative benchmark schemes.

With the rapid development of cloud-network integration, in order to ensure the normal operation of devices, networks, and services in the cloud, anomaly detection algorithms have received increasing attention. The existing anomaly detection methods for multivariate time series, on the one hand, tend to identify situations where multiple metrics deviate significantly from the normal patterns as anomalies, while ignoring anomalies with small deviations from a small number of metrics but very critical to the system, which has a certain tendency. On the other hand, when selecting the window length to extract time series features, it is difficult to take into account both the local and the global data when extracting the data features in the sliding window. Therefore, this paper proposes an anomaly detection method for multivariate time series based on the cross window and probability balance by combining VAE and AE to weaken the tendency of the existing classical algorithm, and can extract the global information and detailed information of the data in the window at the same time. According to the experimental results, the method in this paper has better performance than the best recently baseline, and the F1-score is improved by about 6%.

Open Information Extraction (Open IE) provides a method to extract triplets from text, which has become a research frontier and hot topic in recent years. However, the traditional open information extraction approaches have low precision and significant costs, which heavily rely on the artificially defined extraction paradigm. In this paper, we propose ZoIE, a zero-shot open information extraction model. First, we set positive and negative samples and use the pretraining method based on contrastive learning to train a language model. Then, we use the self-attention weight matrix to perform triplet extraction by using the method of nucleus sampling beam search. Finally, we construct a triplet fine-sorting strategy based on the loss function and select the optimal triplet from the candidate triplets. Experimental results on serveral datasets show that our proposed approach achieves better performance than other baselines.

With the promotion of cloud computing, a large number of hardware and software systems in the cloud bring massive and complex operation and maintenance (O&M) work. To ensure the O&M efficiency of IT infrastructures, it is necessary to implement automatic and reliable scheduling for the directed acyclic graph (DAG) workflow which is composed of multiple O&M tasks. Considering the changing status of networks and machines in the cloud and the position constraints that some tasks must be executed on the specified machines in some O&M scenarios, we propose a novel workflow scheduling approach based on Deep Reinforcement Learning (DRL) to minimize the workflow execution makespan and implement the fault tolerance with the position constraints of tasks execution. In our proposal, we first design a fault-tolerant mechanism according to the reliability requirement and the probability distributions of the machine failure parameters with consideration of different failure rates in the heterogeneous environment. Then, we employ proximal policy optimization (PPO) to optimize the task scheduling strategy and ensure the strategy to satisfy the position constraints of tasks execution by action masking in proximal policy optimization. The experimental results show that our proposal can effectively reduce the makespan of the fault-tolerant workflow on the premise of 99.9% reliability.

Joint extraction of entity and relation is a basic task in the field of natural language processing. Existing methods have achieved good result, but there are still some limitations, such as span-based extraction cannot solve overlapping problems well, and redundant relation calculation leads to many invalid operations. To solve these problems, we propose a novel RelationSpecific Triple Tagging and Scoring Model (RS-TTS) for the joint extraction of entity and relation. Specifically, the model is composed of three parts: we use a relation judgment module to predict all potential relations to prevent computational redundancy; then a boundary smoothing mechanism is introduced to the entity pair extraction, which reallocates the probability of the ground truth entity to its surrounding tokens, thus preventing the model from being overconfident; finally, an efficient tagging and scoring strategy is used to decode entity. Extensive experiments show that our model performs better than the state-of-the-art baseline on the public benchmark dataset. F1-scores on the four datasets are improved, especially on WebNLG and WebNLG∗, which are improved by 1.7 and 1.1 respectively.

With the rapid development of enterprise digital transformation and the widespread use of cloud-native architecture applications, the complexity of enterprise-level systems is getting higher and higher. This requires the engineers to be able to locate and repair alert incidents accurately and quickly to ensure the stability of complex network equipment. In this paper, we propose a novel alert prediction framework, ZAlert, which can predict the occurrence of future alerts and locate the equipment that may cause alerts based on real-time alarm data. First, ZAlert extracts text and statistical features from alarm data to build a high-performance comprehensive learning model to predict the category of alert that may occur in the future with an average performance of 0.81 on F1-score. Then, we use the knowledge graph based on the CMDB(Configuration Management Database) to search for alarm devices. In this way, engineers can quickly find devices that may cause alarms, and improve the the overall efficiency of engineers’ operation and maintenance.

Multiaccess edge computing has emerged as a powerful paradigm for increasing the computation performance of mobile devices (MDs). Applying nonorthogonal multiple access (NOMA) to MEC can further improve the spectrum efficiency and reduce offloading delays caused by the upload congestion. In this article, we examine the joint computation offloading and resource allocation problem in the NOMA–MEC system, which benefits from the combination of NOMA and MEC. Our optimization objective is to minimize the computational overhead (the weighted sum of the execution delay and the energy consumption) in dynamic environments with time-varying wireless fading channels. The optimization problem is formulated as a mixed-integer programming (MIP), which involves jointly optimizing the task offloading decisions, channel assignment, and transmit power allocation. To solve such an optimization problem, we formalize the task offloading and the resource allocation as a Markov decision process (MDP). Then, we propose a deep reinforcement learning (DRL)-based approach, which combines multiple deep neural networks (DNNs) to directly approximate different statistical models for continuous and discrete control. The simulation results demonstrate that the proposed approach can rapidly converge and efficiently decrease the total computational overhead compared to other baseline approaches in different scenarios.

With the rapid development of Server Cluster System technologies, more and more services and tasks are deployed on the Server Cluster Systems. Despite its convenience, a large number of services are mixed together and affect each other, which brings great difficulties to the detection of equipment fault and causes great losses. Nowadays, artificial intelligence for IT operations (AIOps), which utilizes data analysis and machine learning to improve the operation quality of Server Clusters, has been proposed to solve this problem. However, the existing solutions of AIOps cannot be applied to different scenarios. Therefore, in this paper, we propose a novel fault detection method for single indicators which is adaptive to various scenes. To enrich the representation of data, we first propose an interval-volatility-rate method to extract the context features of data in a fixed interval. Based on this, the convolutional neural network and long short-term memory network are employed to get the locally spatial features and temporal features of data. After that, the spatial features and temporal features are combined and input to an MLP network to perform the fault prediction. Additionally, a k − σ principle is designed to promote the sensitivity of fault detection. Experimental results show that our method outperforms other competitive models and has better scalability.

With the rapid increase of users, online meeting platforms have accumulated massive meeting transcripts. However, it is still a challenge for users to quickly master the chief information and manage the meetings, despite there are already some useful text summarization models. In this paper, a Knowledge Graph-based Meeting Summarization Framework is proposed to tackle this challenge. First, a two-layers meeting domain Knowledge Graph is developed to integrate more information of meetings. Based on which, an encoder-decoder architecture is utilized to summarize meetings. For encoding meetings, a structural-level and semantic-level embedding strategy is considered, concretely, the Knowledge Graph is embedded to obtain the structural information, an interaction intention recognition model and a two-level transformer mechanism are devised to get the semantic information. Finally, the structural information and semantic information are combined and fed into the decoding network to generate meeting summaries. Extensive experiments on the Chinese meeting dataset show that our summarization framework outperforms other state-of-the-art models.

With the deepening of the digital transformation, enterprises have accumulated plenty of electronic files. For users, how to quickly find their desired files from the massive resources is a big challenge, which is also a critical concern for enterprises. Although a series of enterprise search engines have emerged in the market, there is a lack of search systems for enterprise files. In this paper, we design a novel enterprise file search framework, named QianXun, which supports searching in both Chinese and English. By introducing technologies such as automatic text summarization, online file previewing, and Knowledge Graph-enhanced search, our proposed framework improves the efficiency of file search as well as the user experience, which is of great significance to enterprises.

Mobile edge computing (MEC) has been regarded as a promising paradigm for increasing the computing capacity of mobile devices (MDs) by offloading tasks to edge servers. Non-orthogonal multiple access (NOMA) is a critical multiple access technique that allows many MDs to transmit on the same resource block simultaneously. Attracted by the enormous benefits of combining NOMA and MEC, we investigate the dynamic computation offloading in a multi-device multi-server NOMA-MEC system. We consider the partial offloading policy such that MDs can offload a portion of the task to the edge server for execution. To minimize the overall computation delay and energy consumption, we formulate a mixed integer programming (MIP) problem to jointly optimize edge server selection and offloading task ratio. Solving the optimization problem with a discrete-continuous hybrid action space is not straightforward since most existing deep reinforcement learning (DRL) algorithms are only applicable to discrete or continuous action spaces. We present the hybrid advantage actor-critic (HA2C) approach, which employs an actor-critic architecture consisting of two par-allel actor networks and a critic network, to tackle this problem. Specifically, the discrete actor and continuous actor networks based on deep neural networks (DNNs) determine MEC server selection and offloading ratio, respectively. The critic network evaluates the current state value, and the advantage function is computed for the discrete and continuous network parameter updates. Experimental results show that the proposed algorithm is superior to conventional DRL algorithms that convert the hybrid action space into a unified homogeneous action space.

With the development of visual programming languages, researchers pay attention to the automatic evaluation of visual projects. Previous work focus on the code evaluation but ignored another essential part—the visualization results. Scratch is a widely used programming platform, and projects created on it are displayed in the form of cartoon clips. It is valuable to explore the visual aesthetics embodied in these clips to fill the gap in the assessment system. We propose a model that predicts the human view scores of cartoon clips created on Scratch. Our method is divided into two steps to evaluate the aesthetic of the sequence images that compose cartoon clips. First, we train an image classification network to predict the relative aesthetics of individual images. Then we construct an aesthetic space for the sequence image and improve the rating within a specific range. We put forward ScratchGAN to generate a Scratch-cartoon-style aesthetic analysis data set for training the classification network. Experimental results show that our Generative Adversarial Network framework can well transform photos into a Scratch-cartoon style. The single image assessment network can generate predictions that fit human cartoon aesthetic opinions. Our method achieves satisfactory results in the aesthetic evaluation of sequence cartoon images.

Nowadays, Scratch, as a widely-used educational programming platform, has gathered a huge number of programming users all over the world. Facing massive programming resources, how to make satisfactory programming recommendations has attracted increasing attention, especially on explainable recommendations. Existing Scratch recommendation systems overlook to provide why a project is recommended, which prevents users from making better decisions and trusting in the system. To resolve this problem, we design the Scratch-RL, an explainable reinforcement learning framework over knowledge graphs for Scratch recommendation. First, we devise a preference-driven Actor-Critic network to simulate users' local preferences and explore the potential interested projects along the reasoning paths. In the Actor-Critic network, we elaborate a preference state function, a preference-based reward function, and a preference-conditional action pruning strategy for the agent. Then, we leverage a directive discriminator network to help evaluate the correctness of recommendations from the agent and return an extra guidance reward accordingly. A high guidance reward is given when the agent generates correct recommendations, which guarantees that the agent quickly and accurately comprehends the preferences of users. Finally, we jointly train the Actor-Critic network and the discriminator, when the whole training is done, the reasoning paths are taken as the interpretability of the recommendations. Extensive experiments on both the Scratch data set and public data set show that, Scratch-RL obtains favorable recommendation results compared with the state-of-the-art models.

With the rapid increase of users, Scratch, as a popular online social and programming platform, has accumulated massive project resources and complex relations across its social and programming learning network. However, it is challenging to utilize the network information for providing Scratch users with personalized services, despite there are already some useful network representation learning models. In this paper, a network representation learning model with preference-based generative adversarial nets for Scratch (ScratchGAN) is proposed to resolve this problem. In ScratchGAN, we first design a node-vector initialization approach to preserve structure information and side information of Scratch network. Then, considering to learn the fine-grained user preference information of network, we propose a novel Scratch adversarial learning model which includes a Scratch generative adversarial net and a user preference difference constraint component. The former aims to capture user preferences through a new generating strategy based on the delivery nature of preference. The latter attempts to embed users' detailed preference differences according to their interaction behaviors. ScratchGAN can mine user preferences while preserving network structure information and side information. Extensive experiments on the Scratch network show that ScratchGAN outperforms other state-of-the-art models in link prediction and recommendation tasks.

Applications consisting of a group of modular tasks can be offloaded to the multiaccess edge computing (MEC) for lower delay and energy consumption. In a dynamic MEC system, the fine-grained cooperative and dynamic offloading solution is necessary for the scenario of reusing tasks among devices. Considering the transmission cooperation, shared wireless bandwidth, and changing task queues on devices and edge servers, we formulate a joint offloading optimization problem to minimize the long-term average task execution cost. Although deep reinforcement learning (DRL) is a popular method for the dynamic problem, existing DRL algorithms are not suitable for our problem because of the hybrid discrete-continuous action spaces and constraints among action dimensions. Therefore, we propose a hybrid average reward proximal policy optimization (hybrid-ARPPO) algorithm to jointly optimize the offloading decisions, cooperative transmission ratios, and edge server assignments. First, we decompose our offloading problem into two subproblems. One is a tractable linear programming problem for continuous transmission ratios, and the other is a Markov decision process (MDP) only with discrete actions for offloading decisions and server assignments. Second, we take the expected average reward as the performance measure and deprecate the discount factor, which can reduce the work of tuning algorithms. Third, we design an action mask layer in the policy network of hybrid-ARPPO to filter invalid actions. Extensive experiments show the effectiveness of our hybrid-ARPPO in different system scales and task arrival patterns.

Mobile-edge computing (MEC) has been a promising solution for Internet-of-Things (IoT) applications to obtain latency reduction and energy savings. In view of the loosely coupled application, multiple devices can use the same task code and different input parameters to obtain diverse results. This motivates us to study the cooperation between devices for eliminating the repeated data transmission. Leveraging coalitional game theory, we formalize the cooperative offloading process of a reusable task into a coalitional game to maximize the cost savings. In particular, we first propose an efficient coalitional game-based cooperative offloading (CGCO) algorithm for the single-task model, and then expand it into a CGCO-M algorithm for the multiple-task model with jointly applying a two-stage flow shop scheduling approach, which helps to obtain an optimal task schedule. It is proved that our CGCO and CGCO-M can achieve the Nash-stable solution with convergence guarantee, and CGCO can obtain an optimal solution. The simulations show that CGCO is equal to the optimal exhaustive search (ES) method and CGCO-M is close to ES in terms of cost ratios. Cost ratios of CGCO and CGCO-M are significantly down by 41.08% and 83.70% compared to local executions, respectively. Meanwhile, CGCO-M obtains 41.46% and 89.74% reductions when reuse factors are 0.1 and 1, which means CGCO-M can save more cost with higher reuse density.

The Software Defined Network in Internet of Things (SDN-IoT) is enjoying growing popularity due to its flexibility, automaticity and programmability. However, there is still a lack of proper permission management on SDN-IoT applications (SIApps), especially when the SIApp’s required northbound interfaces are located in multiple heterogeneous controllers without mutual trust. Existing access control methods are usually based on centralized models, proprietary controllers, trusting conditions or manual operations. It can incur unnecessary performance degradation and poor scalability. To solve this problem, this paper proposes a SIApps’ ledger (SILedger), an open, trusted, and decentralized access control mechanism based on blockchain and attribute-based encryption (ABE). It can not only support effective authorization of SIApps in heterogeneous and untrusted SDN-IoT control domains, but also record all interactions between SIApps and resources, and thus facilitate SIApps further charging, analysis and audit. The main idea is that the SIApps are authorized using access tokens encrypted by ABE, and these tokens are seen as the currency of blockchain to be distributed. Specifically, we re-design blockchain transaction, token encryption, token initialization and token update schemes to achieve cross-domain, fine-grained and flexible SIApps’ permission management. In order to mitigate the delay and complexity problem of blockchain and ABE, we devise an access control framework that separates authorization from call process of SIApps. Finally, we perform security analysis and implement a FISCO-BCOS-based prototype for SILedger. The experimental results show that it can provide effective access control for SIApps with negligible overheads.

The mobile edge computing greatly reduces the transmission delay by offloading computing job to the base station, which is closer to the user. It solves the problem of high delay in cloud computing. Since the edge server has limited resources, improving the efficiency of computation offloading becomes a problem that needs to be addressed. Considering the data is reusable, it is a feasible solution to use the cache to improve the efficiency of computation offloading. In order to make the cache better serve the computation offloading process, we must design a reasonable cache replacement strategy. The traditional cache replacement strategy only considers the cache hit ratio, but ignores the cache value. It is not applicable to the edge computing process. In this paper, a new cache replacement strategy is proposed for the cache replacement problem in mobile edge computing, which is more suitable for computation offloading process. First, we define the cache value based on the priority of computing job. Then, we define the problem of maximizing the cache value in mobile edge computing as a multiobjective optimization problem. Finally, in order to find a suitable Pareto solution, we propose a cache replacement strategy based on the ideal point method. The experimental results show that the proposed algorithm results are significantly better than the existing competitive cache replacement methods.

Nowadays mobile multiplayer games have been an usual recreation, however energy-limited mobile devices confine the game running time. Noticing the component-based multiplayer game is a loosely coupled application whose components are nearly always required among players in one or more rounds, we study the collaboration of players to develop an energy-efficient offloading scheme. We formulate a 0-1 integer nonlinear programming problem to minimize the overall energy cost on player side under time-delay constraint. The problem is intractable to find the optimal solution, thus we propose a heuristic Two-phase Greedy-based Collaborative Offloading Algorithm (TGCOA) for the objective of cost minimization, while including more energy savings for low-energy players. In each round, we preferentially cache components saving more energy per data size for subsequent rounds, then preferentially offload components saving more energy for current round. Meanwhile, we always assign valid player with highest remaining energy to the uploading tasks. Simulations show that the energy cost ratios of our proposal are significantly down by 1.55% to 99.62% compared to four competing methods under different cache limitations and repeatability factors. Meanwhile, our proposal enables a 6.00% to 14.80% lower energy cost proportion for low-energy players compared to the four methods.

In recent years, Scratch has been a popular programming platform for young children. To help children express emotions for projects, Scratch provides children with a music module to create desirable background music. However, in Scratch, there is not a tool helping recognize the emotion of music. Besides, as Scratch-generated music differs from regular music, existing music emotion recognition models perform poor in Scratch-generated music. To overcome it, in this paper, we propose a novel music emotion recognition model for Scratch-generated music. First, we build a Scratch-generated dataset by the main melody extraction algorithm. Then, for each music, we extract their underlying features and input them to the CNN module. After that, the features learned by CNN are input to RNN to get the final classification results. In our model, the CNN module can learn the important features of music while RNN can learn the sequential features. The experimental results show that the proposed model performs better than traditional music emotion recognition models.

The potential for computer games as a tool for AI research and education continues to blossom. The game battle platform provides a place for students to learn artificial intelligence theory and practice artificial intelligence algorithms. However, the existing platforms perform poorly in terms of possessing high concurrency, debugging bot and supporting Scratch language. In order to resolve these problems, this paper first presents a web-API battle mechanism using Websocket protocol. It increases the number of concurrent matches on servers. Next, we give an offline battle tool to make it easier to debug and test bots for users. In addition, the special environment is introduced to support programming bots in Scratch. Finally, we conduct the comparison experiment between the proposed mechanism and traditional battle methods. The results show that it supports higher concurrency and reduces CPU usage by 40%.

Scratch is a popular programming platform with plenty of learning resources. However, it is quite difficult for users to find suitable resources. In this paper, in order to provide the resources which the users require, we propose a Scratch Recommendation Framework Adaptive to User Interest Drifts (SRFA-UID). First, a user interest drifts model is designed, which adopts the time decay factor and the weights of operation behaviors to track users' dynamic interest. Then, on the basis of users' current and historical interest, we calculate their combined user similarity. Next, we present a novel two-hop-algorithm to update users' friend community. Considering the preferences of the whole friend community, the Computational Thinking (CT) skills of users and the impact factors of items, we put forward a recommendation function to obtain items that are related to users. For an item, the function can calculate its F value to determine if we can recommend it to the users. Experimental results show that SRFA-UID performs better than other state-of-the-art methods in the Scratch dataset.

Poetry is a very popular literary form. Currently, its sentiment analysis is one of the hottest research trends. However, there are few relevant studies focusing on the sentiment analysis of ancient Chinese poetry, especially for Tang Poetry. In this paper, we propose a deep learning-based method to solve the above problem. Specifically, we combine Convolutional Neural Network and Gate Recurrent Unit to better extract the characteristics of Tang poetry. In addition, considering the special structural characteristics of Tang poetry, a multi-channel processing model is used to reshape the feature vector of sentences. Finally, in order to verify the rationality and superiority of the proposed methods, we construct a dataset by labeling more than 2500 representative Tang poems. The experimental results prove that our scheme has a higher accuracy rate, up to 64%, compared against three other competing methods.

BlockPy is a block-based program language which has both block-based interface and traditional text-based interface. It fills the gap between block-based programming and language coding. But there is little work that focuses on the Computational Thinking(CT) evaluation of BlockPy programs. In this paper, we design and implement a BlockPy Analysis Tool to assess the CT skills of BlockPy programs automatically. We use Python's built-in AST module to analyse each node in the abstract syntax tree(AST) of each BlockPy program. Then, considering the characteristics of Python language, we propose a new CT Evaluation Criteria based on Scratch Analysis Tool(SAT). Under the guidance of the CT Evaluation Criteria, we propose a detailed scoring program to analyze each node of the program and get the CT score. Experimental results show the superiority of our tool compared with other analysis tools.

In recent years, Scratch has been widely used in helping teenagers learn to program. Before programming with Scratch, users usually need to select a background image with a proper style to set the environment of role activities and emotional tone of works. In order to make the image styles rich and diverse, users can use the fast neural style transfer method to generate the image. However, when users use this method, they are usually puzzled with the optional styles, which makes it a common phenomenon that the contents of transformed images do not match the chosen styles. To resolve this problem, we design a novel image classification model to help recognize the scene of the image, which can guide users to select optional styles for the image. First, we improve the residual network with two attention modules and reconstruct the residual module structure, which improves the accuracy of model classification. Then, we propose a self-adjusting learning rate module, which can accelerate the convergence of the model and reduce fluctuations of the loss function. The experiment results show that our model outperforms other classic image classification methods in classifying the background image of Scratch. The classification accuracy on the testset reaches 98%.

Scratch, a visual programming language used by youth, has received widespread attention of education field. Quality Hound is an effective tool to detect the features of Scratch. However, its detection rules are not sufficiently complete, which incurs incomprehensive results. In this paper, we propose an ANTLR-based feature extraction and detection system to solve this problem. Specifically, nine novel programming feature detection rules are abstracted and applied in our model. The experimental results show our system can effectively extract programming features from projects and provide feedback for students and teachers.

Software defined networks have been a driving force of the Internet of Things (IoT) advancement in the devices management and network control. One of the important challenges remaining to be resolved is how to safely complete the network update with low resources' consumption. In this paper, we propose a new batch-level update mechanism with low cost (BLLC) to solve this problem. Its main idea is to bundle control commands and update networks from the destination to the source of new flows. Specifically, we first build update trees for all new flows according to the network states and consistency properties. Then, the new rules in an update tree are packaged into a single control packet (UBCP). Furthermore, a virtual destination-based algorithm is devised to minimize the number of finally formed UBCPs. The generated UBCPs will successively pass all IoT nodes to be updated, and instruct them to apply the required operations. If the control channel is wireless, we introduce a cooperative transmission mechanism to enhance the reliability of the UBCPs' forwarding. Finally, we evaluate our scheme by comprehensive experiments. The results show that the BLLC reduces the link cost by 68% on the average with a slight performance loss in the update time.

One of the crucial technologies for future Internet of Things (IoT) is software defined networks, which provides a centralized and programmable control ability for operators. However, the current single control plane deployed on the remote IoT gateway may incur a bottleneck with the continuous growth of IoT devices and applications. In this paper, we propose a two-level hierarchy (the master and slave) control framework to resolve this problem. Additionally, a novel slave controller placement strategy (SCPS) is presented to further optimize the control performance. In SCPS, we first design a synthetic IoT node importance assessment model based on an improved analytic hierarchy process and fuzzy integral. It considers the device attributes, service attributes, and control frequency. Then we formulate the slave controller placement as a binary integer program problem. It is solved by a modified binary particle swarm optimization algorithm to optimize the control delay and control cost of critical IoT nodes. Finally, we carry out extensive experiments to evaluate the performance of our strategy. The results show that, compared to other competing methods, it approximately reduces the control delay of important IoT nodes by 30.56%.

With the introduction of computer programming in schools around the world, Scratch has risen in prominence for its thinkable, meaningful and social. Aiming to assessing the Computational Thinking skills of a Scratch program, we design and implement a new Scratch program analysis tool based on ANTLR. To solve some flaws (e.g., high failure rate and low efficiency) in Dr. Scratch which is the most relevant tool to assess Computational Thinking skills of Scratch programs, we choose the recognition tool ANTLR to design the system module and the assessing flow. And then, we customize more than 200 lexical and syntax parser rules in ANTLR. Furthermore, we expand the grading standard of assessing Computational Thinking skills in Dr. Scratch. Some fundamental concepts in Computer Science, such as stack, queue and recursion method, are involved in our grading standard. Experiment results show the performance (e.g., success rate, execution time) of SAT is superior to that of Dr. Scratch.

Nowadays the integration of IoT and SDN has been a research hotspot which attracts significant attention. However, as resources are relatively limited in IoT, direct application of SDN will cause some challenges, one of them is that IoT forwarding nodes cannot store massive complex FlowTables like a traditional OpenFlow switch. To solve the problem, this paper proposes a hash-based distributed storage strategy. Specifically, we present a multi-dimension selection method to decide the optimum distributed storage location. And then a hash space is formed by using the content of data flow in IoT, it is the basis of FlowTables deployment and data forwarding in distributed storage. Moreover, we introduce a new FlowTables search mechanism which is built on the principle of the binary tree. Experimental results demonstrate that our strategy efficiently improves the FlowTables storage capacity with small performance loss in IoT.

Since Apple introduced the iBeacons in Worldwide Developers Conference (WWDC) 2013, the iBeacon has been rapidly accepted and generalized in the market. For the deployed iBeacons, it is necessary to monitor their status. In this paper, we design a crowd sensing based monitoring framework which combines the moving and static schemas of participants to monitor the real status of iBeacons. In such a system, the inaccuracy and conflict of the collected signal information, commonly caused by the error rate of participants or the differences of sensing context, have received more and more attention. Estimating the real status of iBeacons according to the uploaded signal information becomes a big challenge for our monitoring system. Towards this end, we propose a context-aware estimation approach in this paper. We first model the effects of sensing context, and then propose an iterative method to infer the error rate of participants and estimate the real status of iBeacons with high precision. Our method is tested via extensive simulations, and verified by our monitoring system which has been applied in the teaching building. The results demonstrate that the proposed estimation approach outperforms recent popular three-estimates algorithm and OtO EM algorithm. At last, we develop the review mechanism, which ensures the efficiency of our monitoring system.

For crowd sensing applications, the inaccuracy and conflict of the reported samples, commonly caused by untrained participants and heterogeneous mobile devices, have received more and more attention. Given this, constructing a high-quality radio map of iBeacons via crowd sensing is very challenging. In this paper, we resort to the family of robust estimation and design a robust radio map construction scheme, named by RrMCS, to enhance the quality of iBeacon map construction via crowd sensing. For heterogeneous devices, we use a linear robust estimation technique to map the signal strength scanned by different users into a uniform space. Moreover, we alleviate the negative influence of outliers and detect the abnormal participants by utilizing the multivariate robust statistics on the signal information uploaded by different users. Finally, we implement an Android-based system for constructing the iBeacon radio map and conduct extensive experiments under indoor environment. The experimental results validate the enhancement of the constructed iBeacon radio map by the proposed RrMCS.

Indoor localization is an important primitive that can enable many ubiquitous computing applications. This paper improves the scheme of landmark and inertial navigation through crowd sensing to address reliable and accurate indoor localization. The location of landmark can calibrate the location of user and inertial navigation can optimize the location of landmark in turn. In this work, we define the landmark as certain characteristic structure with Beacon. To tackle the challenges of misjudgment of landmarks and variability in user walking profiles, we have developed algorithms for reliable detection of landmarks and personalization of step length with the aid of crowd sensing. We have built an indoor localization system integrating these modules and an indoor floor map, which can be further improved with more users using our system. We demonstrate for the first time a meter-level indoor localization system that is self-improving, user adaptive, and easy to deploy. Extensive experiments on users with smartphone devices, with over 37 subjects walking over an aggregate distance of over 20 kilometers were carried out. Evaluation results show that our system can achieve a mean accuracy of 2m initially and 1m with the calibration of landmarks in a 39m × 21m testing area.

Indoor localization is an important primitive that can enable many ubiquitous computing applications. In this paper, we choose the iBeacon as landmark and improve the localization scheme of iBeacon and inertial navigation through crowd sensing. Specifically, we use crowd sensing to design a parameter learning algorithm for device diversity. Furthermore, we take advantage of crowd sensing to collect the correction information of iBeacon about inertial navigation, which can optimize the step length estimation and direction inference. We demonstrate for the first time a meter-level indoor localization system that is self-improving, user adaptive, and inclusive to diversity. Extensive experiments on users with mobile devices, with over 37 subjects walking over an aggregate distance of over 10 kilometers were carried out. Evaluation results show that the accuracy is within 2m in a 39m×21m testing area.

It is common to take photos and share them by smartphone in mobile social network at present, but the sharer may need to mask partial areas of the photo to avoid leaking the privacy. Reversible image mosaic can mask the privacy very well. However, there is little discussion on how to effectively control and safely store the mosaic information for recovering. In this paper, we first propose a prediction algorithm of reversible image mosaic size (PARIMoS), which can calculate the maximum number of mosaic blocks according to the image quality of non-mosaic area and the recovery quality of mosaic required by user. And then, we propose an efficient and reliable generating algorithm of reversible image mosaic (GAoRIM). In GAoRIM, we compress the secret information of the mosaic area with twice-run-length-encoding and then store the compressed result in special non-mosaic areas of image separately. Therefore, it can ensure reliable recovery even the image is partly stained, and hardly change the roughness of non-mosaic area. The experiment results show that the biggest change of average roughness in the non-mosaic area is less than 0:5%. Comparing with the similar algorithms, on average, the amount of data protected by GAoRIM is at least 8:70% higher than theirs while the peak signal noise ratio (PSNR) of stego image without mosaic is similar to theirs.

Indoor localization is an important primitive that can enable many ubiquitous computing applications. This paper improves the scheme of landmark and inertial navigation through crowd sensing. The location of landmark can calibrate the position of users, and accurate localization can optimize the landmark's location in turn. In this work, we define landmarks as certain characteristic structures with iBeacons. To tackle the challenge of low efficiency of a landmark, we combine Bluetooth signals and sensor readings to reflect the distinctive signature of a landmark and then design a method for a reliable detection of a landmark. Moreover, we investigate the calibration bias of a landmark to ensure the calibration accuracy. For improving the accuracy of inertial navigation, we personalize the step length model and correct the user's heading with the aid of crowd sensing. We have built an indoor localization system integrating the above modules and an indoor floor map, which can be further improved with more users using our system. We demonstrate for the first time a meter-level indoor localization system that is self-improving, user adaptive, and easy to deploy. Extensive experiments on users with mobile devices, with over 37 subjects walking over an aggregate distance of over 30 km, were carried out. Evaluation results show that our system can achieve a mean accuracy of 2 m initially and 1 m with the calibration of landmarks in a 39 m × 21 m testing area. Copyright © 2016 John Wiley & Sons, Ltd.

In participatory sensing applications, the inaccuracy and conflict of the reported data, commonly caused by the error rate of participants or the differences of observation context, have received more and more attention. Estimating the real status of facilities according to observations becomes a big challenge for participatory sensing. Towards this end, we propose a context-aware estimation approach based on participatory sensing in this paper. We first model the effects of observation context, and then propose an iterative method to infer the error rate of participants and estimate the real status with high precision. Our method is verified using a public facilities monitoring application in our campus, and tested via extensive simulations. The results demonstrate that the proposed method outperforms recent popular three-estimates algorithm and OtO EM algorithm.

In this paper, we propose a habit-based SWRL generation and reasoning approach in smart home. Definition and recognition of habits of daily living can provide humanized smart home for assisted living application, especially for people with memory deficits. This paper presents Recognizing Habit of Daily Living(RHDL) by discovering and monitoring smart home context information. The habit and habit association of using electrical appliances are defined explicitly for the first time. The generation rules between habit/complex habit and SWRL are designed, and the reasoning is based on the Semantic Web Rule Language(SWRL). The ontology model for the RHDL is designed and the prototype system of RHDL is implemented using protege and Jess tools.

In smart building systems, the automatic control of devices relies on matching the sensed environment information to customized rules. With the development of wireless sensor and actuator networks (WSANs), low-cost and self-organized wireless sensors and actuators can enhance smart building systems, but produce abundant sensing data. Therefore, a rule engine with ability of efficient rule matching is the foundation of WSANs based smart building systems. However, traditional rule engines mainly focus on the complex processing mechanism and omit the amount of sensing data, which are not suitable for large scale WSANs based smart building systems. To address these issues, we build an efficient rule engine. Specifically, we design an atomic event extraction module for extracting atomic event from data messages, and then build a β-network to acquire the atomic conditions for parsing the atomic trigger events. Taking the atomic trigger events as the key set of MPHF, we construct the minimal perfect hash table which can filter the majority of the unused atomic event with O(1) time overhead. Moreover, a rule engine adaption scheme is proposed to minimize the rule matching overhead. We implement the proposed rule engine in a practical smart building system. The experimental results show that the rule engine can perform efficiently and flexibly with high data throughput and large rule set.

Smart building systems can provide flexible and configurational sensing and controlling operations according to users' requirements. As the number and the complexity of service rules customized by users have significantly increased, there is an increasing danger of conflict during the interaction process between users and the system. To address this issue, we propose a new rule conflict detection scheme tailored for the smart building system. First, we present a formal rule model UTEA based on User, Triggers, Environment entities, and Actuators. This model can handle not only controlled devices with discrete status but also real-valued environmental data such as temperature and humidity. In addition, this model takes multiple users with different authorities into account. Second, we define 11 rule relations and further classify conflicts into five categories. Third, we implement a rule storage system for detecting conflicts and design a conflict detection algorithm, which can detect the conflict between two rules as well as cycle conflict/multicross contradiction among multiple rules. We evaluated our scheme in a real smart building system with more than 30 000 service rules. The experiment results show that our scheme improves the performance in terms of error/missed-detection rates and running time.

In MANET, the cooperative authentication mechanism requires the cooperation of the neighbor nodes and significantly enhances the authentication probability. However, it exposes location privacy of neighbor nodes and is costly. How to balance the authentication and location privacy is a key issue. In this paper, we use game theory to analyze the behavior of neighbor nodes in cooperative authentication and gain the optimal strategy. Every node seeks to obtain most reward at least location privacy loss and cost. We first build the static game with complete information and obtain two pure-strategy and one mixed-strategy Nash equilibria. These equilibria can be used efficiently to balance authentication and location privacy. Then, we build the static game with incomplete information and obtain the Bayesian Nash equilibria.

Mobile social networks (MSNs) are a kind of delay tolerant network that consists of lots of mobile nodes with social characteristics. Recently, many social-aware algorithms have been proposed to address routing problems in MSNs. Because of the social properties introduced to routing, this results in node privacy disclosure. In this paper, analyzing social-based routing strategies, we propose a privacy attack tree model taking all the possible attack on social-based routing into account. This model describes all of the possibilities and approaches of privacy disclosure, and quantifies their the occurrence probability.

Clustering sensor nodes is an effective method for achieving high energy efficiency and flexible data aggregation in wireless sensor networks. However, scheduling all clusters to avoid the inter-cluster interference and meanwhile send all observation to the sink quickly is a difficult problem. In this paper, we use interference graph to model the inter-cluster collision problem and propose an efficient sleep scheduling mechanism in the clustered wireless sensor networks. We firstly divide the clusters into layers, then transform the cluster-tree network into the interference graph of clusters, and thereby divide all clusters on one layer into different independent sets. All clusters in the same independent set can work simultaneously without interference, while all independent sets are scheduled one by one for collision-free. To reduce the total delay of data report in the whole network, we propose three kinds of rules to sort the independent sets. Simulation and analysis show that the degree-based sorting rule works best. The practical experiments also verify this conclusion.

In Wireless Multimedia Sensor Networks (WMSNs), data fusion and collaborative in-network processing operations often require effective multimedia synchronization control. Extensive researches have been done in the traditional networks. Most of these works assume that there exists a powerful synchronization controller in the network. However, for WMSNs, the in-network processing of the multimedia content is usually performed based on the resource-constrained sensors. Traditional synchronization control algorithms fail to run on the low-end hardware platform. In this paper, we propose an adaptive synchronization control scheme with multi-level buffer to address this problem. Main contributions of this paper are as follows. (i) To overcome the resource constraint of WMSNs, we perform the synchronization control in a distributed way by combining the sensors in the stream transmission path. (ii) By evaluating the current network state, we adjust the level number of the buffer adaptively to balance the delay and delay jitter. (iii) Through analyzing the buffer state of each level, we propose an efficient packet scheduling scheme to reduce the delay jitter actively. (iv) We implement the proposed scheme and verify its effectiveness in our practical WMSNs platform. The experiment results show that the proposed scheme can maintain the streams synchronization effectively with the low-end sensors.

For wireless multimedia sensor networks (WMSNs) deployed in noisy and unattended environments, it is necessary to establish a comprehensive framework that protects the accuracy of the gathered multimedia information. In this paper, we jointly consider data aggregation, information trust, and fault tolerance to enhance the correctness and trustworthiness of collected information. Based on the multilayer aggregation architecture of WMSNs, we design a trust-based framework for data aggregation with fault tolerance with a goal to reduce the impact of erroneous data and provide measurable trustworthiness for aggregated results. By extracting statistical characteristics from different sources and extending Josang's trust model, we propose how to compute self-data trust opinion, peer node trust opinion, and peer data trust opinion. According to the trust transfer and trust combination rules designed in our framework, we derive the trust opinion of the sink node on the final aggregated result. In particular, this framework can evaluate both discrete data and continuous media streams in WMSNs through a uniform mechanism. Results obtained from both simulation study and experiments on a real WMSN testbed demonstrate the validity and efficiency of our framework, which can significantly improve the quality of multimedia information as well as more precisely evaluate the trustworthiness of collected information.

Wireless Sensor Networks are attracting lots of attention for its broad application areas, and hence the services provided by WSNs become much more diverse. But with the development of in-depth applications, service discovery and composition are challenges to the end users. Service mining is a new technique in Internet which can discover new composite services from the abundantly existing services in Internet to meet the users' needs. However, those service mining methods are not practical in the energy-exhausted and application-oriented WSNs. In this paper, we develop a semantics-based service mining method for WSNs, to provide users with interesting composite services. In this method, services are combined and recommended to users actively according to the calculation of service similarity and an up datable semantic database. By the results of service similarity, useless compositions are filtered out so that energy consumption on service flooding can be reduced. The update strategy of semantic database is also given out, by which the composite services can keep up with time and be more applicative. The benefits of the proposed method are that all operations such as calculating, filtering, and updating are simple enough and can be performed at broker nodes.

In Wireless Audio Sensor Networks (WASNs), many applications such as target tracking, data fusion and localization, often require precise synchronization among audio streams. Extensive researches have been done under the assumption that wireless channel is well enough and the cluster head is power supplied. However, these conditions can not be satisfied in many applications. When audio event occurs, the burst of data is usually coupled with a high packet loss rate. Besides, the cluster head is usually battery-powered, especially in the wild environment. In this paper, we propose a distributed audio synchronization scheme to address these challenges. By dynamically detecting the endpoint of the audio streams and marking the start point of the audio synchronization obviously at the reporting sensor, we can eliminate the sound propagation delay effectively and synchronize the audio streams precisely even under the high packet loss rate. Moreover, by handling the synchronization task in a distributed way, we can lighten the burden of the cluster head and prolong the lifetime of the whole network greatly. To improve the flexibility of the scheme, we also propose a self-adaptive algorithm which can adjust the parameters of the synchronization according to the network condition. Experimentally, we show that the proposed scheme can synchronize the audio streams under different conditions with low energy cost.

Video surveillance system, which can provide real-time display of the monitored scene and video playback, has been employed in many areas including: commercial security, accident investigation, law enforcement and emergency response. However, audio which carries important information not available in video is usually not taken seriously and used effectively. In this paper, we develop an enhanced surveillance platform by introducing the low-power wireless audio sensor networks (WASNs). We can obtain more comprehensive and precise monitoring without the limitation of the line-of-sight and lighting condition. Moreover, this platform is designed and built for providing key support to varieties of applications. This article describes the platform architecture, including design, implementation, and performance. We describe the audio sensor platform which can deliver high-quality audio over sensor network by multi-hops with low power requirement. In addition, we present the multimedia synchronization mechanism in the heterogeneous network which is the foundation of applications in the proposed platform. Our experiments include an in-depth analysis of the bottlenecks within the platform as well as measurements for the various components.

Wireless Audio Sensor Networks (WASNs) can provide event detection, object tracking and audio stream monitoring through cooperative audio sensor nodes. Extensive researches have focused on sound detection and source localization, but little work is on the monitoring and recovery of audio stream. To clearly reconstruct the real-time audio stream from WASNs, both the quantity and quality of samples provided by sensor nodes should be carefully considered. In this work, we propose a novel approach termed "adaptive sampling and diversity reception", which can significantly reduce the energy consumption of network while maintaining high accuracy of audio stream. On one hand, by selectively picking sampling nodes and adaptively adjusting their sampling rates, sufficient samples arrive at the fusion node for high recovering fidelity. On the other hand, through Maximal-Ratio-Combining(MRC)-like signal fusion scheme at the fusion node, the signal-to-noise ratio (SNR) of combined audio is further improved. Meanwhile, we propose a heuristic algorithm of sampling rate assignment which can select a set of optimal sampling nodes and hence minimize the energy consumption on data transmission. Analytically and experimentally, we show that the proposed scheme can improve the signal recovering fidelity tremendously with low energy cost.

Wireless Audio Sensor Networks (WASNs) can provide event detection, object tracking and emergency response through cooperative audio sensor nodes. Effective collaboration of audio sensors requires precise synchronization among audio streams. Some researches have been done on the timestamp mechanism based on time synchronization ignoring propagation delay and many other researches have focused on the synchronization of simple gunshot or scream. However, for the synchronization of intermittent and fluctuating audio stream, there still exists many challenges. In this paper, we propose an effective audio synchronization scheme which can synchronize the intermittent audio streams adaptively while maintain low energy cost. On one hand, we obtain audio synchronization without global clock which save energy tremendously. On the other hand, by introducing a feedback loop mechanism, we can keep a high audio synchronization fidelity even when the audio source moves around and the sound strength varies with time. Furthermore, we discuss the extension for flexibility and scalability of this scheme when there exist several sound sources simultaneously or the audio source moves among clusters. Through experiments on a WASNs platform and simulations, we show that the proposed scheme is desirable to guarantee the accuracy of audio synchronization in practical environment with low energy cost.

Acoustic sensor networks can provide rich multimedia information, which doubtlessly benefit a plethora of data hungry applications such as environment monitoring, health care, emergency response, and security surveillance. In this paper, we study the data unit delivery process of cluster head by developing a discrete-time Markov chain model and a M/G/1 queue model, based on a single-hop cluster-based sensor networks architecture. Our analysis shows how arrival rate and the number of sensor nodes influence mean waiting time and how many sensor nodes that a cluster head can support. The result of our study can be used to guide the design of real-time acoustic wireless sensor networks system.