The high complexity and wide applications of Cyber-Physical Systems (CPSs) pose a large requirement on both accuracy and interpretability of the time-series anomaly detection algorithms. While a large number of deep learning algorithms have achieved excellent accuracy, the interpretability is often limited, especially when considering retaining correlations in multivariate time-series. In this paper, we propose a novel multivariate time-series anomaly detection method based on adaptive masking mechanism to improve both accuracy and interpretability, which contains a specially designed series saliency module. For more intuitive and interpretable results, a learnable adaptive mask is introduced in the series saliency module, which can disclose the influence on anomalies in both feature and temporal dimensions. The original time-series and their versions with adaptive perturbations added are then mixed via the mask forming an adaptive data augmentation method to improve the accuracy of anomaly detection. Furthermore, the anomaly detection module is model-agnostic, whether based on forecasting or reconstruction. The optimization of the training objectives will lead to more accurate and interpretable detection results. With four real-world datasets, we demonstrate that the adaptive mask can provide more accurate anomaly detection results with meaningful interpretations in the form of a mask matrix.

The prevalence of smart devices encourages increasing requirements of wearable human–computer interactions. To improve user acceptance, such interactions require easy-to-manipulate and unobtrusive characteristics. In this article, we, for the first time, propose to recognize silent commands through a lightweight and around-ear biosensing system Mordo that can be easily integrated with earphones, manipulate smart devices, and minimize social awkwardness. In particular, we first determine the empirical principles of constructing commands and experimentally screen the commands based on the around-ear configuration. Second, we select the optimal around-ear sensor configuration according to the single-channel signal-to-noise ratios (SNRs) and classification accuracies. Third, we propose a multistream CNN-LSTM network to learn the spatiotemporal mapping between the around-ear signals and commands. Finally, extensive experiments have been conducted to evaluate the feasibility and stability. The results indicate an averaged accuracy of 89.66% that outperforms other algorithms of similar tasks. The stability tests show that our system presents sufficient stability under command deformations and head motions. We demonstrate the necessity of collecting such scale of data by gradually reducing training data size. We also validate the generalization ability of our method toward other sensing parameters by reducing the spatial and temporal resolutions. The proof-of-concept design will aim the further development of the commercial products for silent command recognition.

Real-time surveillance systems have become a necessity of today's life owing to their relevance in the contemporary era for security reasons to ensure a secure and safe environment. Presently, Person re-identification (Re-ID)-based surveillance systems are becoming increasingly more prevalent and sophisticated since they do not require human intervention and are more reliable to deploy in public spaces leveraging multi-camera networks. However, one of the major problems in Person ReID is the visual appearance i-e the appearance of a person in an image is greatly affected by different camera views. As a result, the discriminative set of features must be learned in a deep learning model in order to re-identify persons from opposing camera viewpoints. To address this challenge, we propose an image/text-retrieval-based Person ReId method in which both visual and text-based features are exploited to carry out person re-identification. More precisely, the textual descriptions of the images are taken into account as text features with Glove Word Embedding followed by 1D-MAPCNN and fused with image-level features extracted using the GoogLeNet model. In addition, the feature discriminability is enhanced using local importance-based pooling (LIP) layers in which adaptive significance weights are learned during downsampling. Moreover, from two different modalities, feature refinement is done during training with the help of attention mechanisms using the Convolutional Block Attention module (CBAM) and the proposed shared attention neural network. It is observed that LIP layers along with both vision and textual features are playing a key role in acquiring discriminative features even if the visual appearance of the same person is greatly affected due to camera pose conditions. The proposed method is validated on the CUHK-PADES dataset and has 15.34% and 24.39% rank-1 improvement in text and image-based retrievals.

Currently, the internet of everything (IoE) enabled smart surveillance systems are widely used in various fields to prevent various forms of abnormal behaviors. The authors assess the vulnerability of surveillance systems based on human gait and suggest a defense strategy to secure them. Human gait recognition is a promising biometric technology, but one significantly hindered because of universal adversarial perturbation (UAP) that may trigger system failure. More specifically, in this research study, the authors emphasize on sample convolutional neural network (CNN) model design for gait recognition and assess its susceptibility to UAPs. The authors compute the perturbation as non-targeted UAPs, which trigger a model failure and lead to an inaccurate label to the input sample of a given subject. The findings show that a smart surveillance system based on human gait analysis is susceptible to UAPs, even if the norm of the generated noise is substantially less than the average norm of the images. Later, in the next stage, the authors illustrate a defense mechanism to design a secure surveillance system based on human gait.

Daily peak load forecasting (DPLF) is critical in smart grid applications for security analysis, unit commitment, and scheduling of outages and fuel supplies. Although excellent single machine learning methods using tree-based ensemble learning or deep learning have shown satisfactory performance for DPLF, there is still room for improvement. This study proposes a hybrid tree-based ensemble learning model, called HYTREM, for robust DPLF. We first collected two commercial buildings’ energy consumption data from publicly available datasets. We then performed data preprocessing, such as input variable configuration, for the HYTREM modeling. We divided both datasets into training and test sets and generated the prediction values of several tree-based ensemble learning models, such as gradient boosting machine, extreme gradient boosting, Cubist, and random forest (RF), for each set as novel input variables. We reconstructed datasets using the Boruta algorithm to select all the relevant features and built an online RF model trained on these datasets using time-series cross-validation for day-ahead DPLF. The experimental results showed that the HYTREM performed a better performance than tree-based ensemble and deep learning methods in building-level DPLF in terms of the mean absolute percentage error and normalized root mean square error.

The mobile traffic data and resources using IoE in wireless networking have raised numerous problems in terms of performance monitoring in edge-connected devices [1]. Next-generation networks, such as 6G and cybertwin, are implemented to address these problems. Sixth-generation (6G) communication would play a vital role in supporting complex wireless interconnectivity. In order to allow millions of connected devices and applications to operate smoothly at high data rates and low latency, a network of the 6G is anticipated [2]. The only access point for the Internet is cybertwin, which serves as a contact hub and tracks all user requirements. In the edge-cloud cyberspace, cybertwin is a digital database of smartphone activities, terminals, objects, etc. The integrated use of technology such as blockchain, 6G, and cybertwin is a multidisciplinary area for designing effective and efficient IoE systems [3]. The purpose of this Special Issue is to examine the new technology, innovative architectures, and future problems in depth in terms of network secured infrastructure based on cybertwin for 6G-enabled IoE.

Deep learning has revolutionized the field of computer vision and image processing. Its ability to extract the compact image representation has taken the person re-identification (re-id) problem to a new level. However, in most cases, researchers are focused on developing new approaches to extract more fruitful image representation and use it in the re-id task. The extra information about images is rarely taken into account because the traditional person re-id datasets usually do not have it. Nevertheless, the research in multimodal machine learning has demonstrated that the utilization of the information from different sources leads to better performance. In this work, we demonstrate how a person re-id problem can benefit from the utilization of multimodal data. We have used the UAV drone to collect and label the new person re-id dataset, which is composed of pedestrian images and its attributes. We have manually annotated this dataset with attributes, and in contrast to the recent research, we do not use the deep network to classify them. Instead, we employ the continuous bag-of-words model to extract the word embeddings from text descriptions and fuse it with features extracted from images. Then the deep neural decision forest is used for pedestrians classification. The extensive experiments on the collected dataset demonstrate the effectiveness of the proposed model.

Deep learning methods, e.g., convolutional neural networks (CNNs) and Recurrent Neural Networks (RNNs), have achieved great success in image processing and natural language processing especially in high level vision applications such as recognition and understanding. However, it is rarely used to solve information security problems such as attack detection studied in this paper. Here, we move forward a step and propose a novel multi-channel intelligent attack detection method based on long short term memory recurrent neural networks (LSTM-RNNs). To achieve high detection rate, data preprocessing, feature abstraction, and multi-channel training and detection are seamlessly integrated into an end-to-end detection framework. Data preprocessing provides high-quality data for subsequent processing, then different types of features are extracted from the processed data. Multi-channel processing is used to generate classifiers by training neural networks with different types of features, which preserve attack features of input vectors and classify the attack from normal data. With the results of the classifier's attack detection, we introduce a voting algorithm to decide whether the input data is an attack or not. Experimental results validate that the proposed attack detection method greatly outperforms several attack detection methods that use feature detection and Bayesian or SVM classifiers.

The ultimate receiver of image and video is human visual system (HVS). It is an important problem in the domain of image and video processing that how to establish visual information representation model meeting the HVS perception property. In this study, authors give theory analysis and experiment results to prove that l_1 norm‐based entropy of primitive (EoP) is superior to the l_0 norm‐based EoP for the monocular cue in image quality assessment. By developing the concept of mutual information of primitive (MIP) as the binocular cue, an l_1 EoP‐based stereoscopic image quality assessment metric is proposed. With EoP as monocular cue and MIP as binocular cue, the relative entropy between the original stereoscopic image and the distorted one is explored to predict the quality score with support vector regression. To avoid destroying image's structured information, the structured EoP (SEoP) is further explored to measure the stereoscopic image information. Extensive experimental results demonstrate that the stereoscopic image quality assessment algorithm with SEoP as monocular cue and MIP as binocular cue outperforms many state‐of‐the‐art ones.

A multi-representative re-fusion (MRRF) approximate data collection approach is proposed in which multiple nodes with similar readings form a data coverage set (DCS). The reading value of the DCS is represented by an R-node. The set near the Sink is smaller, while the set far from the Sink is larger, which can reduce the energy consumption in hotspot areas. Then, a distributed data-aggregation strategy is proposed that can re-fuse the value of R-nodes that are far from each other but have similar readings. Both comprehensive theoretical and experimental results indicate that the MRRF approach increases lifetime and energy efficiency.

As we delve into the Internet of Things (IoT), we are witnessing the intensive interaction and heterogeneous communication among different devices over the Internet. Consequently, these devices generate a massive volume of Big Data. The potential of these data has been analyzed by the complex network theory, describing a specialized branch, known as 'Human Dynamics.' The potential of these data has been analyzed by the complex network theory, describing a specialized branch, known as 'Human Dynamics.' In this extension, the goal is to describe human behavior in the social area at real-time. These objectives are starting to be practicable through the quantity of data provided by smartphones, social network, and smart cities. These make the environment more intelligent and offer an intelligent space to sense our activities or actions, and the evolution of the ecosystem. To address the aforementioned needs, this paper presents the concept of 'defining human behavior' using Big Data in SIoT by proposing system architecture that processes and analyzes big data in real-time. The proposed architecture consists of three operational domains, i.e., object, SIoT server, application domain. Data from object domain is aggregated at SIoT server domain, where the data is efficiently store and process and intelligently respond to the outer stimuli. The proposed system architecture focuses on the analysis the ecosystem provided by Smart Cities, wearable devices (e.g., body area network) and Big Data to determine the human behaviors as well as human dynamics. Furthermore, the feasibility and efficiency of the proposed system are implemented on Hadoop single node setup on UBUNTU 14.04 LTS coreTMi5 machine with 3.2 GHz processor and 4 GB memory.

The rate of data generation is enormously growing due to the number of internet users and its speed. This increases the possibility of intrusions causing serious financial damage. Detecting the intruders in such high-speed data networks is a challenging task. Therefore, in this paper, we present a high-speed Intrusion Detection System (IDS), capable of working in Big Data environment. The system design contains four layers, consisting of capturing layer, filtration and load balancing layer, processing layer, and the decision-making layer. Nine best parameters are selected for intruder flows classification using FSR and BER, as well as by analyzing the DARPA datasets. Among various machine learning approaches, the proposed system performs well on REPTree and J48 using the proposed features. The system evaluation and comparison results show that the system has better efficiency and accuracy as compare to existing systems with the overall 99.9 % true positive and less than 0.001 % false positive using REPTree.

No abstract available.

Multimedia has been exponentially increasing as the biggest big data, which consist of video clips, images, and audio files. Processing and analyzing them on a cloud data center have become a preferred solution that can utilize the large pool of cloud resources to address the problems caused by the tremendous amount of unstructured multimedia data. However, there exist many challenges in processing multimedia big data on a cloud data center, such as multimedia data representation approach, an efficient networking model, and an estimation method for traffic patterns. The primary purpose of this article is to develop a novel tensor-based software-defined networking model on a cloud data center for multimedia big-data computation and communication. First, an overview of the proposed framework is provided, in which the functions of the representative modules are briefly illustrated. Then, three models,—forwarding tensor, control tensor, and transition tensor—are proposed for management of networking devices and prediction of network traffic patterns. Finally, two algorithms about single-mode and multimode tensor eigen-decomposition are developed, and the incremental method is employed for efficiently updating the generated eigen-vector and eigen-tensor. Experimental results reveal that the proposed framework is feasible and efficient to handle multimedia big data on a cloud data center.

The Machine-to-Machine (M2M) communication has the potential to connect millions of devices in the near future. Since they agree on this potential, several standard organizations need to focus on improved general architecture for M2M communications. Currently, there is a lack of consensus to improve the general feasibility of M2M communication. Heterogeneous Mobile Ad hoc Networks (HetMANETs) can normally be considered appropriate for M2M challenges. When a mobile node (MN) moves inside a HetMANET, various challenges including a selection of the target network and energy efficient scanning take place, which need to be addressed for efficient handover. To cope with these issues, we propose a handover management scheme that efficiently initiates a handover process and selects an optimal network. Our proposed scheme is composed of two phases, i.e., i) the MN performs handover triggering based on the optimization of the Receive Signal Strength (RSS) from an access point/base station (AP/BS), and, ii) the network selection process is carried out by considering different parameters such as delay, jitter, velocity, network load, and energy consumption by the network interface. Moreover, if there are more networks available, then the MN selects the one that can provide the highest quality-of- service (QoS) using the Elimination and Choice Expressing Reality (ELECTRE) decision model. The performance of the proposed scheme is compared in the context of the number of handovers, average stay-time of an MN in the network, and energy consumption against periodic and adaptive scanning. Similarly, a two- state Markov model is defined that efficiently distribute the number nodes on the available access points and base stations. The proposed scheme efficiently optimizes the handoff related parameters and outperforms existing schemes.

ABSTRACT In this paper, we propose a new security log management scheme for smart surveillance in a multi‐camera environment. Basically, our security log consist of descriptions for various behavior properties of moving objects, such as motion type, time, and speed in a merged camera view. To generate such security log, we first analyze the input video frame from each surveillance camera and construct a motion vector of interest points in the frame. By analyzing the motion vector, we recognize moving objects and trace their local behavior in the video. On the basis of this analysis, we can calculate various global behavior features of the objects in the merged camera view, which can be acquired by stitching together the frames from multiple camera inputs. Such global behavior features are captured into security logs, which can be used to smartly carry out various surveillance operations such as retrieving objects whose behavior is similar to a query behavior or whose behavior shows predefined abnormality. Because our scheme treats all the objects in the frame independently, it can handle multiple objects simultaneously. We implemented a prototype system and performed various experiments to demonstrate that our scheme can achieve a reasonable performance. Copyright © 2013 John Wiley & Sons, Ltd.

ABSTRACT We present a new 3D object tracking algorithm that supports multiple planar and nonplanar objects with real‐time processing speed and high accuracy. The main problem of object tracking algorithm is the limitation of the supporting type of target object, slow processing speed, and low tracking accuracy. Our algorithm provides high accuracy and real‐time performance while detecting not only planar objects but also nonplanar objects. The real‐time performance is accomplished by using Features from Accelerated Segment Test corner detection, region of interest, and parallel processing on a multicore processor. High accuracy is realized by using a scale‐invariant feature transform descriptor, random sample consensus, region of interest, and double robust filtering. Copyright © 2013 John Wiley & Sons, Ltd.

In this article, we present a wearable intelligence device for activity monitoring applications. We developed and evaluated algorithms to recognize physical activities from data acquired using a 3-axis accelerometer with a single camera worn on a body. The recognition process is performed in two steps: at first the features for defining a human activity are measured by the 3-axis accelerometer sensor and the image sensor embedded in a wearable device. Then, the physical activity corresponding to the measured features is determined by applying the SVM classifier. The 3-axis accelerometer sensor computes the correlation between axes and the magnitude of the FFT for other features of an activity. Acceleration data is classified into nine activity labels. Through the image sensor, multiple optical flow vectors computed on each grid image patch are extracted as features for defining an activity. In the experiments, we showed that an overall accuracy rate of activity recognition based our method was 92.78%.

Web service composition is the process of constructing a set of Web services which, when invoked with some user input in a particular order, can produce the output to the user's requirements. This paper proposes a novel model checking based approach for automated service composition. Modeling services as a set of interleaved processes in a class of process algebra, we formulate service composition as model checking asserted on a specific type of property on the model. We show that, under this formulation, correct composition workflows can be constructed from the counter-examples provided by model checking. With a case study on online hotel booking services, we demonstrate that the proposed approach can support directed a cyclic composition graphs and the generated composition graphs are automatically verified.

Semantic web services (SWS) provide rich and formal representations of services by capturing the semantics of requests and service descriptions as well as the context of service interactions. Recently, multiple SWS standards such OWL-S, WSMO, and SAWSDL have emerged making service interoperability increasingly a challenge. In order to leverage the true potential of Semantic Web services, this service heterogeneity problem must be overcome. This paper introduces an OWL-S based mediator for service level and data level mediation across heterogeneous semantic service descriptions. We consider WSMO and SAWSDL service description formalisms and present mediation strategies via OWL-S, by converting service descriptions and mapping the underlying core ontology from WSML to OWL. Through experiments, the proposed mediator is found to be promising.

The authors present an approach that extracts human activity patterns of daily living and represents spatiotemporal relations between activities intuitively. In general, customised services are provided based on activity patterns of users. This study focuses on extracting and determining activities that occur simultaneously. In order to determine simultaneous activities, the authors analysed the daily activities that are collected from device applications such as location sensors and electronics. In addition, a context model using the incremental statistical method is organised and temporal relations between the activities patterns are analysed. Furthermore, information visualisation of the spatiotemporal topology with duration and frequency is demonstrated. Also, the authors have experimented on a test-bed called the ubiquitous smart space and compared the accuracy of the incremental statistical method with that of the non-incremental method.

A typical music clip consists of one or more segments with different moods and such mood information could be a crucial clue for determining the similarity between music clips. One representative mood has been selected for music clip for retrieval, recommendation or classification purposes, which often gives unsatisfactory result. In this paper, the authors propose a new music retrieval and recommendation scheme based on the mood sequence of music clips. The authors first divide each music clip into segments through beat structure analysis, then, apply the k-medoids clustering algorithm for grouping all the segments into clusters with similar features. By assigning a unique mood symbol for each cluster, one can transform each music clip into a musical mood sequence. For music retrieval, the authors use the Smith-Waterman (SW) algorithm to measure the similarity between mood sequences. However, for music recommendation, user preferences are retrieved from a recent music playlist or user interaction through the interface, which generates a music recommendation list based on the mood sequence similarity. The authors demonstrate that the proposed scheme achieves excellent performance in terms of retrieval accuracy and user satisfaction in music recommendation.

In this paper, we discuss the use of knowledge for analyzing and retrieving music contents semantically. First, we present Context-based Music Recommendation (COMUS) ontology to reason desired user emotion state from context and user preference information in the ontology. COMUS is a music dedicated ontology in OWL constructed by incorporating domain specific classes for music recommendation into the Music Ontology, which include situation, mood and musical features. More specifically, we describe the ontologies of mood and situation in music using low-level features like pitch or duration and musical factors like tempo or rhythm. Our proposed ontology defines generic as well as domain-specific concepts whose detection is important for the analysis and description of music in a specific domain. As a novelty, our ontology can express detailed and complicated relations among the music, moods and situations, enabling users to find appropriate music for the music retrieval and recommendation application. We present some of the experiments we performed as a case-study for music recommendation.

With the advent of the ubiquitous era, context-based music recommendation has become one of rapidly emerging applications. Context-based music recommendation requires multidisciplinary efforts including low level feature extraction, music mood classification and human emotion prediction. Especially, in this paper, we focus on the implementation issues of context-based mood classification and music recommendation. For mood classification, we reformulate it into a regression problem based on support vector regression (SVR). Through the use of the SVR-based mood classifier, we achieved 87.8% accuracy. For music recommendation, we reason about the user's mood and situation using both collaborative filtering and ontology technology. We implement a prototype music recommendation system based on this scheme and report some of the results that we obtained.

Query reformulation has been suggested as an effective way to improve retrieval efficiency in text information retrieval and one of the well-known techniques for query reformulation is user relevance feedback. Recently, there has been an increased interest in the query reformulation using relevance feedback with evolutionary techniques such as genetic algorithm for multimedia information retrieval. However, these techniques have still not been exploited widely in the field of music retrieval. In this paper, we propose a novel music retrieval scheme that is based on user relevance feedback with genetic algorithm and evolutionary method with neural network. The former is for reformulating a user query and the latter is for reducing the population size by learning neural network. We implemented a prototype music retrieval system called MUSEMBLE based on this scheme. Experimental results showed that our proposed scheme achieves a good performance.

Accurate voice humming transcription and efficient indexing schemes are essential for a large-scale humming-based music retrieval system. Although many researches have been done to develop such schemes, their performances are not still satisfactory. In our previous works, we proposed (i) a new voice query transcription scheme [4], (ii) a popularity-adaptive indexing structure called FAI [6] for fast retrieval, and (iii) a semi-supervised relevance feedback and query reformulation scheme based on a genetic algorithm [7] in order to improve retrieval efficiency. In this demonstration, we extend our efforts to a mobile environment and develop a prototype mobile music retrieval system called M-MUSICS. Our focus in this implementation includes versatile user interface for easy querying and browsing on a typical mobile device such as PDA phone and satisfactory performance in a wireless mobile environment. We report some of the results.