Large Language Models (LLMs) have recently emerged as a leading approach for multivariate time series forecasting. However, their effectiveness is hampered by a fundamental architectural mismatch: the permutation-invariant self-attention of Transformers lacks inductive biases for the strict temporal order and complex cross-variable dependencies inherent in time series. Existing methods often sidestep this issue with input-level alignment techniques rather than endowing the model itself with structural awareness. To address this gap, we introduce GraFT (Graph-infused Forecasting Transformer), a framework that systematically embeds relational priors into a pre-trained backbone by constructing a heterogeneous patch relation graph, which represents both universal temporal principles with static edges and instance-specific patterns with dynamic adaptive edges. To process this multi-relational structure, a relational graph convolutional network generates structure-aware representations, which are infused into the patch embeddings to provide explicit structural guidance to the Transformer's attention mechanism. Extensive experiments show that GraFT achieves state-of-the-art performance on long-term forecasting and zero-shot learning, outperforming leading LLM-based methods on eight standard benchmarks with an average Mean Squared Error (MSE) reduction of 14.4%.

edge intelligence | internet of autonomous unmanned vehicles | software systemsWith the development of the embedded systems, navigation, sensors, robots, and big data analytics, the automobile vehicles industry has been one major economic sector recently, and its economical and societal impacts continue to expand.To extend the capabilities of automobile vehicles, the Internet of Autonomous Unmanned Vehicles (IAUV) has been proposed to form a global network of sensors, robots and unmanned vehicles, improving cooperation between heterogeneous communication systems to provide reliable Internet services in civil applications such as environmental monitoring, video surveillance, network provisioning, wireless power transfer, and emergency or disaster assistance.However, these powerful applications always require the support from automobile, transportation, wireless communications, networking, resource management, intelligent computing, security, and robotics, and so on.Due to diverse and interdisciplinary nature of these challenges, architectures, algorithms and developmental software systems proposed by networking, robotics, transportation, cognitive and artificial intelligence research communities will need to be utilized.Fortunately, the abilities of IAUV systems can be greatly realized with high performance of low latency and high reliability by applying artificial intelligence-enabled computing, communication, edge service deployment and flexible resource schedule of unmanned vehicles, robots and sensors appropriate and ---------------

Abstract Recent developments in the field of Internet of things (IoT) have aroused growing attention to the security of smart devices. Specifically, there is an increasing number of malicious software (Malware) on IoT systems. Nowadays, researchers have made many efforts concerning supervised machine learning methods to identify malicious attacks. High‐quality labels are of great importance for supervised machine learning, but noises widely exist due to the non‐deterministic production environment. Therefore, learning from noisy labels is significant for machine learning‐enabled Malware identification. In this study, motivated by the symmetric cross entropy with satisfactory noise robustness, the authors propose a robust Malware identification method using temporal convolutional network (TCN). Moreover, word embedding techniques are generally utilised to understand the contextual relationship between the input operation code (opcode) and application programming interface function names. Here, considering the numerous unlabelled samples in real‐world intelligent environments, the authors pre‐train the TCN model on an unlabelled set using a word embedding method, that is, Word2Vec. In the experiments, the proposed method is compared with several traditional statistical methods and more recent neural networks on a synthetic Malware dataset and a real‐world dataset. The performance comparisons demonstrate the better performance and noise robustness of their proposed method, especially that the proposed method can yield the best identification accuracy of 98.75% in real‐world scenarios.

Coronavirus disease-2019 (COVID-19) as a new pneumonia which is extremely infectious, the classification of this coronavirus is essential to effectively control the development of the epidemic. Pathological changes in the chest computed tomography (CT) scans are often used as one of the diagnostic criteria of COVID-19. Meanwhile, deep learning-based transfer learning is currently an effective strategy for computer-aided diagnosis (CAD). To further improve the performance of deep transfer learning model used for COVID-19 classification with CT images, in this article, we propose a hybrid model combined with a semi-supervised domain adaption model and extreme learning machine (ELM) classifier, and the application of a novel multikernel correntropy induced loss function in transfer learning is also presented. The proposed model is evaluated on open-source datasets. The experimental results are compared to some baseline models to verify the effectiveness, while adopting accuracy, precision, recall,$F_{1}$score and area under curve (AUC) as the evaluation metrics. Experimental results show that the proposed method improves the performance of original model and is more suitable for CT images analysis.

In this paper, we present a simulation study of a Proof of Stake (PoS) consensus algorithm used in a public blockchain called Nxt. We first provide an overview of Nxt and its PoS consensus algorithm design, and introduce a concise mathematical analysis of the Nxt PoS algorithm. We then present an experimental study on the Nxt PoS behavior in a simulated environment and in a small network running Nxt private nodes. We show that the fractions of the blocks generated by forgers in the system are generally proportional to their weight in stakes provided that the weight is relatively small, which is consistent with the mathematical analysis. We consider two scenarios of double-spending attacks: (1) a single dominating forger with large stakes; and (2) a group of colluding forgers that collectively control large stakes. The simulation results show that the single-forger attack is more advantageous over using a pool of forgers in launching successful double-spending attacks. Finally, we propose a quota-based mechanism to limit the fraction of blocks that any forger could generate. We show that the mechanism is highly effective in mitigating the single-forger attack, but has limited success in preventing the double-spending attacks based on forging pools.

Abstract With the rapid development of communication technologies, the quality of our daily life has been improved with the applications of smart communications and networking, such as intelligent transportation and mobile service computing. However, high user demands for quality of service (QoS) are forcing intelligent transportation to continuously improve immediacy and reduce the tasks offloading delay for the internet of vehicles (IoV). To meet the low latency of vehicle tasks offloading, an offloading scheme combining mobile edge computing (MEC) and deep reinforcement learning (DRL), is proposed in this article. Firstly, a realistic map is simulated, while initializing the tasks queue and building a tasks offloading environment with multiple service nodes. Then, an algorithm that combines deep learning with reinforcement learning, that is, the deep Q‐learning network (DQN) algorithm, is developed to optimize the offloading scheme by reducing the offload latency. Finally, given that the complete information cannot be observed effectively in the environment, a long short‐term memory (LSTM) model is applied within the DQN to train its neural network to improve offloading efficiency. The simulation results show that the MEC‐based vehicle tasks offloading can effectively reduce the latency of vehicle offloading.

No abstract available.

In this paper, we present a study on person identification using static features extracted from Kinect skeleton data. On the contrary to previous reports that the dynamic features such as gait parameters are more discriminative than static features, we find that by using a combination of a set of easy to obtain static features, we can achieve nearly perfect accuracy in identifying persons with only a few frames. In our study, we experimented with several classifiers, including k-nearest neighbor (KNN), decision tree, Gaussian Naive Bayesian, neural network with multiplayer perception (MLP), and support vector machine (SVM), and several combinations of static skeleton features. In all scenarios, KNN outperforms other classifiers consistently. MLP and SVM require a huge amount of parameter tuning and training time and they do not perform well compared with KNN except for small gallery sizes when all static features available.

Storage environments designing and planning is the prerequisite of Automated Guided Vehicles (AGVs) route planning in Automated Storage and Retrieval Systems (AS/RSs), and could reduce much labor force and the energy cost with an automatic way. Although image processing has made great progress, few publications focused on establishing rasterized storage environments. In this paper, we proposed a novel method for automatically designing and planning storage environments based on a monocular camera. First, we take a picture of the vacant warehouse ground with a monocular camera and then obtain the rectangular image using the perspective transformation. Second, we draw the approximate boundary of the storage area and compute the external enclosing rectangle as the exact boundary. Third, the computer divides the environment into several grids based on the size of workstations. Finally, the computer labels the obstacle workstations according to the planning results. The proposed approach is validated with several simulations.

The scheduling efficiency for Automated Guided Vehicles (AGVs) is one of the key factors that affect the work efficiency of Automated Storage and Retrieval Systems (AS/RSs). The scheduling environments are more complex than traditional scenarios because there are more static obstacles, and the optimal solutions may be different under the same condition. First, we present two AGVs scheduling policies based on Euclidean distance and Manhattan distance. Then, we propose the improved Dijkstra’s algorithm-based scheduling policy by introducing a penalty term in classical Dijkstra’s algorithm. We further analyze the advantages and disadvantages of above three scheduling policies. Finally, by validating the presented algorithms through simulations, we propose the improved Dijkstra’s algorithm-based policy is the least-energy-cost scheduling policy for rasterized warehouse environments.

In this paper, we present the design and implementation of a blockchain-enabled secure sensing data processing and logging system. Although in our implementation we use the IOTA distributed ledger, the main mechanisms introduced in the system are independent from any particular blockchain platform for maximum flexibility. A blockchain platform is used as an immutable datastore to store critical data for secure sensing data processing and logging. This system corporates several innovative mechanisms. First, a sensor identification mechanism is put in place to ensure that only sensing data submitted by eligible sensors are accepted. Second, only aggregated sensing data are transmitted to the blockchain for safe-keeping. This strategy has two benefits: (1) it reduces the throughput requirement on the public blockchain; and (2) it saves on the transaction fees for asking the blockchain to store the data. Furthermore, a mechanism is introduced to allow two-level logging using a local datastore in conjunction with the blockchain to extend the immutability property offered by the blockchain to the locally stored raw sensing data. Third, a set of mechanisms are designed to facilitate the query of the blockchain for a particular subset of the aggregated data, and to ensure that given an aggregated data item, the corresponding raw data can be quickly located, which is essential for data retrieval needed when situations arise such as forensic analysis in cases of incidents and for auditing purposes. The system is implemented in the Python programming language and a preliminary testing on the functionality of the system has been conducted.

In this article, we provide a concise but systematic review on blockchain-enabled cyber-physical systems (CPS). We dissect various blockchain-enabled CPS as reported in the literature in terms of their operations and the features of blockchain that have been used. We identify key common CPS operations that can be enabled by blockchain, and classify them in terms of their time sensitivity and throughput requirements. We also elaborate and classify features of blockchain in terms of different levels of benefits to CPS, including security, privacy, immutability, fault tolerance, interoperability, data provenance, atomicity, automation, data/service sharing, and trust. Finally, we point out two primary open research issues for developing blockchain-enabled CPS, namely, excessive delay in reaching consensus and limited throughput, and outline future research directions.

As the application of the industrial Internet of Things (IIoT) becomes more widespread, the IIoT is being combined with social networks. Nodes in the network can be users, machines, and so on. Using the sensing detection technology of the IIoT, industrial machines can realize real-time informatization, which is convenient for users to perform remote management. Nodes can communicate with each other and make ratings. These ratings can be modeled as directed weighted edges between nodes and form directed weighted networks (DWNs). The edge weight represents the “strength” of relationship and the direction of edge points from the edge generator to the edge receiver. Predicting edge weights in DWNs is critical to predicting unknown ratings or recovering lost data. In this article, we propose a directed edge weight prediction model (DEWP) using decision tree ensembles. It extends the local similarity indices to DWNs and extracts a series of similarity indices between nodes as features of each edge. These features are used to construct a blended regression model of random forest, gradient boost decision tree, extreme gradient boosting, and light gradient boosting machine. The proposed algorithm was evaluated experimentally with the Bitcoin OTC and Bitcoin Alpha datasets by removing 10% to 90% of edges in the original network. Compared with other classical algorithms, DEWP has higher prediction accuracy and robustness.

The papers in this special section focus on multi-modal computing in biomedical applications. n recent years, the development of biomedical imaging techniques, integrative sensors, and machine learning, brings many benefits to the diagnosis of various diseases. We can collect, measure, and analyze vast volumes of health-related data using the technologies of computing and networking, leading to tremendous opportunities for the health and biomedical community. Meanwhile, these technologies have also brought new challenges and issues. Biomedical intelligence, especially precision medicine, is considered one of the most promising directions for healthcare development.

In this paper, we present an exploratory work towards the recognition of activities and performing real-time objective assessment in human patient simulation (HPS). Although HPS has been pervasively used in medical and nursing programs in developed countries, there is a huge need in providing consistent and objective assessment on student performance during HPS. Current methods all depend on instructor subjective observation, which not only could lead to inconsistency in evaluation across different students and different instructors, but also are very time and resource intensive. Recognizing complex human activities in the context of HPS is very challenging because it involves the recognition of human actions, gestures, as well as human-object and human-mannequin interactions. Hence, we study the feasibility of developing such a system for a particular simulation where a student is required to first identify the patient and then place a neck brace on the patient's neck. The system we that we have developed identifies the actions and activities in the simulation and provides qualitative assessment on the student performance using computer vision, OpenPose, and TensorFlow. The system also consists of a debriefing mobile app that the student and instructor could use to view an automatically generated report with supporting key frames captured and annotated by our system.

The research on three-dimensional topology is important for Internet of Thing. Small-world with shorter average path lengths has proven to be an effective model for building evolutional network topologies. In order to build three-dimensional topology in IoT, the ant colony algorithm is used to plan shortcuts in this paper. First, Gaussian integration is used to simulate the ups and downs of terrain in three-dimensional space. Second, a significant number of nodes are randomly deployed on the modeled terrain. Taking into account the information about slope and aspect around the node, the actual sensing range of the node is calculated. Third, a certain percentage of nodes are selected as super sensor nodes. Finally, the ant colony algorithm is used to add shortcuts between super sensor nodes. Extensive experimental results show that an energy-efficient three-dimensional network topology in IoT can be built by the algorithm.

In recent years, more and more wearable sensors have been employed in smart health applications. Wearable sensors not only can be used to collect valuable health-related data of their users, they can be also used in conjunction with other infrastructure-bound sensors, such as Microsoft Kinect sensor, to facilitate privacy-aware fine-grained activity tracking. This fusion of multimodal data promises a new type of smart health applications that coach a user to live a healthier life style by monitoring the user in realtime and reminding him or her when he or she engages in an unhealthy activity. In this paper, we investigate how to achieve fine-grained activity recognition in the context of such an application. In our scheme, the identification accuracy is improved by incorporating a nonlinear and local similarity measure, namely kernel risk-sensitive loss, into a novel multilayer neural network learning algorithm, called as stacked extreme learning machine. Furthermore, to achieve a good generalization performance with minimal human intervention, Jaya as a popular optimization algorithm, is also used to adjust key parameters in our proposed approach. The experiments are conducted to verify the effectiveness of the proposed scheme.

In this paper, we introduce a novel mechanism to facilitate automatic user authentication for privacy-aware vision based human activity tracking. The mechanism relies on the Bluetooth beacon technology, which localizes a user who is present in the field of view of a camera. Unlike our previous mechanism designed for the same purpose, which requires a user to push a button on the smartwatch he or she is wearing and make a predefined gesture to register with the activity tracking system, this new mechanism does not require the user to alter his or her work routine when entering the view of the tracking system. Hence, the proposed mechanism significantly increases the usability of the tracking system. While the importance of automatic user authentication might not be obvious to researchers, it is essential for the acceptance of the activity tracking technology in its intended venues where workers will be monitored on their jobs as demonstrated by our previous field study and by our interviews with business owners. We report the experimental result based on two types of Bluetooth beacon devices, one from Estimote, and the other from Gimbal. Through the experiments, we identify several challenges in using these commercial-off-the shelf beacon devices for automatic user authentication, including the lack of synchronized beacon signal transmission by different beacon devices, nonuniform beacon transmission with occasional large gaps, and delay in beacon signal detection or reporting, and propose solutions to these issues.

In this paper, we report our work on applying an artificial neural network (ANN) to interpret brain wave signals into words. Signals were acquired by a four-channel human electroencephalography (EEG) head set. EEG data were recorded through a video-guided user interface with time stamps. The objective of the experiment is to set up a two-word-based EEG library and to predict a random trail of participants' mental activities, in terms of words. We show that our algorithms can achieve a 90% prediction accuracy of the Bernoulli experiments.

Recently, wind speed forecasting as an effective computing technique plays an important role in advancing industry informatics, while dealing with these issues of control and operation for renewable power systems. However, it is facing some increasing difficulties to handle the large-scale dataset generated in these forecasting applications, with the purpose of ensuring stable computing performance. In response to such limitation, this paper proposes a more practical approach through the combination of extreme-learning machine (ELM) method and deep-learning model. ELM is a novel computing paradigm that enables the neural network (NN) based learning to be achieved with fast training speed and good generalization performance. The stacked ELM (SELM) is an advanced ELM algorithm under deep-learning framework, which works efficiently on memory consumption decrease. In this paper, an enhanced SELM is accordingly developed via replacing the Euclidean norm of the mean square error (MSE) criterion in ELM with the generalized correntropy criterion to further improve the forecasting performance. The advantage of the enhanced SELM with generalized correntropy to achieve better forecasting performance mainly relies on the following aspect. Generalized correntropy is a stable and robust nonlinear similarity measure while employing machine learning method to forecast wind speed, where the outliers may exist in some industrially measured values. Specifically, the experimental results of short-term and ultra-short-term forecasting on real wind speed data show that the proposed approach can achieve better computing performance compared with other traditional and more recent methods.

The Acknowledgements published in the original article contain errors. The correct version is given below.

In this paper, we report our experiences in designing, deploying, and making continuous improvements of a Privacy-Aware Compliance Tracking System (PACTS) at a skilled nursing facility. The purpose of PACTS is to help state tested nursing assistants (STNAs) get into the habit of using proper body mechanics when performing bedside cares. The system has been deployed in six resident rooms and seven STNAs have been participating our study for over ten weeks. This study makes the following contributions: (1) A registration mechanism that enables an STNA to register with any of the rooms that have PACTS installed, which is essential to protect the privacy of patients and non-participating persons; (2) A wrong activity detection mechanism that is robust against occlusions due to furniture and against temporary inability of floor determination; (3) A lease-based mechanism to improve the usability of PACTS, which allows an STNA to be continuously monitored without having to register with PACTS repeatedly when she/he goes in and out of the view of the Kinect sensor for the duration of the lease.

Poor form of body mechanics has been attributed to as a major risk factor for lower back injuries, which costs billions of dollars a year in the US alone. In this paper, we report a case study on using a Kinect-based system during an annual competency training at a local nursing home to promote safe resident handling. Each participant of the study was asked to perform three specific bedside care tasks while being monitored by our system, and he/she was provided with realtime feedback in the form of a vibration via smart watch that he/she worn on detection of a wrong activity by our system. At the end of the session, the participant was asked to complete a short survey regarding the performance of our system and his/her opinion about the system usability. There are two major findings in this case study: (1) the majority of the nursing assistants engaged in poor body mechanics frequently when performing the designated tasks, which indicated that traditional training is not rigorous and may fail to accomplish its purpose, and (2) most participants expressed positive attitude towards using our system for competency training as well as during their jobs to reduce the risk of injuries.

The total US box office revenue exceeds ten billion dollars a year. Inevitably, new product forecasting and diagnosis have high financially stakes. The motion picture industry has a huge incentive to perform early prediction of movie success or failure. In this paper, we present a study that evaluate viewer responses to short movie trailers using a cap instrumented with Electroencephalogram (EEG) sensors. The method can be used to evaluate prerelease movies regarding how engaging they are to viewers. The primary advantage of our approach is that responses of a viewer can be recorded without any manual inputs from the viewer. The data collected, if analyzed properly, can reveal more accurate information regarding the viewer's emotion states while watching the movie than post-viewing surveys. This approach also enables the delivery of personalized entertainment through the brain interface.

Among the many cell types involved in epileptogenesis, the GABAergic interneurons are of great interests, largely due to their context-dependent role in controlling cellular excitability and dynamic behavior in seizure. By controlling GABAergic interneuron activity with optogenetic technology, we sought to block seizure activity in the hippocampus in a 4-aminopyridine induced seizure model in mice. This paper introduces an experiment where intracranial electroencephalogram (EEG) was acquired from the hippocampal CA3 area during seizure while the GABAergic interneurons were optically altered. EEG segments during GABAergic interneurons inhibition (Laser ON) was compared to the condition when GABAergic interneurons were not inhibited by the light (Laser OFF). We utilized an adaptive Amplitude Correlation Threshold metric to define certain Seizure Level as a key parameter for epileptic detection. We also quantified the degree of inhibition as an evaluation parameter for the laser-interfered treatment. We found that both the frequency and strength of the ictal events were evidently reduced when the GABAergic interneurons were optically inhibited. This work reveals a novel functional role of GABAergic interneurons in seizure.

In this paper, we present a security and privacy enhancement (SPE) framework for unmodified mobile operating systems. SPE introduces a new layer between the application and the operating system and does not require a device be jailbroken or utilize a custom operating system. We utilize an existing ontology designed for enforcing security and privacy policies on mobile devices to build a policy that is customizable. Based on this policy, SPE provides enhancements to native controls that currently exist on the platform for privacy and security sensitive components. SPE allows access to these components in a way that allows the framework to ensure the application is truthful in its declared intent and ensure that the user's policy is enforced. In our evaluation we verify the correctness of the framework and the computing impact on the device. Additionally, we discovered security and privacy issues in several open source applications by utilizing the SPE Framework. From our findings, if SPE is adopted by mobile operating systems producers, it would provide consumers and businesses the additional privacy and security controls they demand and allow users to be more aware of security and privacy issues with applications on their devices.

Lost productivity from lower back injuries in workplaces costs billions of U.S. dollars per year. A significant fraction of such workplace injuries are the result of workers not following best practices. In this paper, we present the design, implementation, and evaluation of a novel computer-vision-based system that aims to increase the workers' compliance to best practices. The system consists of inexpensive programmable depth sensors, wearable devices, and smart phones. The system is designed to track the activities of consented workers using the depth sensors, alert them discreetly on detection of noncompliant activities, and produce cumulative reports on their performance. Essentially, the system provides a valuable set of services for both workers and administrators toward a healthier and, therefore, more productive workplace. This study advances the state of the art in the following ways: 1) a set of mechanisms that enable nonintrusive privacy-aware selective tracking of consented workers in the presence of people that should not be tracked; 2) a single sign-on worker identification mechanism; 3) a method that provides realtime detection of noncompliant activities; and 4) a usability study that provides invaluable feedback regarding system design and deployment, as well as future areas of improvements.

In this article, we present a set of lightweight mechanisms to enhance the dependability of a safety-critical real-time distributed system referred to as an integrated clinical environment (ICE). In an ICE, medical devices are interconnected and work together with the help of a supervisory computer system to enhance patient safety during clinical operations. Inevitably, there are strong dependability requirements on the ICE. We introduce a set of mechanisms that essentially make the supervisor component a trusted computing base, which can withstand common hardware failures and malicious attacks. The mechanisms rely on the replication of the supervisor component and employ only one input-exchange phase into the critical path of the operation of the ICE. Our analysis shows that the runtime latency overhead is much lower than that of traditional approaches.

Kernel least-mean mixed-norm (KLMMN) algorithm as a special kernel adaptive filter method achieves good performance when the measured noises are distributed with a linear combination of long-tails and short-tails. In order to reduce the computational efforts and improve the accuracy, this paper proposes a novel entropy optimized kernel learning algorithm, called E-KLMMN, on the basis of information entropy and KLMMN. The first step of E-KLMMN algorithm is to calculate the entropy weights of input vectors in the training set which contains the linear combination of long-tailed and short-tailed distribution noises. Then we remove the input vectors and their corresponding outputs whose entropy weights are less than the average value. Finally, using the modified training set to train KLMMN model, the following data points thus could be predicted. Through the use of information entropy, the proposed algorithm E-KLMMN has the advantages of high precision and low cost, while employing it to noise environment. We use the actual data to conduct the experiment, and the comparisons among E-KLMMN, KLMS, and KLMMN demonstrate the effectiveness and superiority of our algorithm.

Extreme learning machine (ELM) as a learning algorithm for neural networks (NN) could provide the best generalization performance at extremely fast leaning speed. Through the use of ELM, it is thus possible to improve the existing schemes especially the ones whose learning speed is not fast enough while addressing control problems. As a popular NN-based approach for control applications, direct heuristic dynamic programming (DHDP) with a good capability of adaptive learning has been successfully applied to solve control problems. But limited by slow learning algorithms in NN, it imposes very challenging obstacles to the real-time controller design of DHDP, which keeps it from widely applied. In this paper, driven by the interest of improving learning speed of DHDP while maintaining its good approximation performance, we employ ELM as a learning algorithm in DHDP. The proposed ELM-based DHDP learning scheme is tested on a cart-pole balancing control problem. The simulation results show the proposed scheme has better learning performance than traditional DHDP. Furthermore, this paper provides a novel idea of applying ELM in control problems.

In this paper, we present a novel system designed to promote healthy living at home. The system integrates Microsoft Kinect and wearable devices such as smart watches and fitness bands to enable selective tracking of user activities at the home setting. The objective of the system is to continuously monitor each user and detect bad postures that could increase the risk of back injuries, and prolonged sedentary bouts that are not conducive for a healthy lifestyle. The wearable device to be worn by each user also delivers realtime feedback to the user on detection of bad postures or inactivities. Furthermore, activities data are logged for each individual at a home server and can be assessed via mobile devices or regular Web browsers.

In this paper, we present the design and implementation details of a Kinect-based framework for human activity tracking. The framework is intentionally designed to be open so that it can communicate over the network with other systems and mobile/wearable devices. The possibility of integrating with other devices and systems makes it possible to use Kinect for human activity tracking in a way unforeseen before. For example, the integration of our framework with wearable sensors, such as smart watches and fitness bands, enables us to perform selective tracking of the daily activities of a particular user and provide realtime feedback to the user. Furthermore, multiple frameworks could work together to form a federated system to cover a large area and/or a large number of users.

Microsoft Kinect, a low-cost motion sensing device, enables users to interact with computers or game consoles naturally through gestures and spoken commands without any other peripheral equipment. As such, it has commanded intense interests in research and development on the Kinect technology. In this paper, we present, a comprehensive survey on Kinect applications, and the latest research and development on motion recognition using data captured by the Kinect sensor. On the applications front, we review the applications of the Kinect technology in a variety of areas, including healthcare, education and performing arts, robotics, sign language recognition, retail services, workplace safety training, as well as 3D reconstructions. On the technology front, we provide an overview of the main features of both versions of the Kinect sensor together with the depth sensing technologies used, and review literatures on human motion recognition techniques used in Kinect applications. We provide a classification of motion recognition techniques to highlight the different approaches used in human motion recognition. Furthermore, we compile a list of publicly available Kinect datasets. These datasets are valuable resources for researchers to investigate better methods for human motion recognition and lower-level computer vision tasks such as segmentation, object detection and human pose estimation.

Event stream processing has been used to construct many mission-critical event-driven applications, such as business intelligence applications and collaborative intrusion detection applications. In this paper, we argue that event stream processing is also a good fit for autonomic computing and describe how to design such a system that is resilient to both hardware failures and malicious attacks. Based on a comprehensive threat analysis of event stream processing, we propose a set of lightweight mechanisms that help achieve Byzantine fault tolerant event processing for autonomic computing. The mechanisms consist of voting at the event consumers and an on-demand state synchronization mechanism triggered when an event consumer fails to collect a quorum of matching decision messages. We also introduce an evidence-based safe-guarding mechanism that prevents a faulty event consumer from inducing unnecessary rounds of state synchronization.

Byzantine fault tolerance has been intensively studied over the past decade as a way to enhance the intrusion resilience of computer systems. However, state-machine-based Byzantine fault tolerance algorithms require deterministic application processing and sequential execution of totally ordered requests. One way of increasing the practicality of Byzantine fault tolerance is to exploit the application semantics, which we refer to as application-aware Byzantine fault tolerance. Application-aware Byzantine fault tolerance makes it possible to facilitate concurrent processing of requests, to minimize the use of Byzantine agreement, and to identify and control replica nondeterminism. In this paper, we provide an overview of recent works on application-aware Byzantine fault tolerance techniques. We elaborate the need for exploiting application semantics for Byzantine fault tolerance and the benefits of doing so, provide a classification of various approaches to application-aware Byzantine fault tolerance, and outline the mechanisms used in achieving application-aware Byzantine fault tolerance according to our classification.

Mobile devices have experienced explosive growth and rapid adoption. These devices have also become troves of security and privacy data of the consumers that utilize them. What makes mobile devices unique from traditional computing platforms is the additional sensing components they contain and their ease of access which allow consumers to make these devices a part of their lives. Additionally these devices are fragmented in operating systems, sensing capabilities, and device manufacturers. In this paper we define an ontology that can be utilized as a foundation for enforcing security and privacy policies across all mobile devices, and use the ontology to define policies and to model knowledge elements for mobile devices. We also identify areas where the policies can be applied, including whether to enforce policies on the device or in the cloud.

The low latency fault tolerance (LLFT) system provides fault tolerance for distributed applications within a local-area network, using a leader–follower replication strategy. LLFT provides application-transparent replication, with strong replica consistency, for applications that involve multiple interacting processes or threads. Its novel system model enables LLFT to maintain a single consistent infinite computation, despite faults and asynchronous communication. The LLFT messaging protocol provides reliable, totally ordered message delivery by employing a group multicast, where the message ordering is determined by the primary replica in the destination group. The leader-determined membership protocol provides reconfiguration and recovery when a replica becomes faulty and when a replica joins or leaves a group, where the membership of the group is determined by the primary replica. The virtual determinizer framework captures the ordering information at the primary replica and enforces the same ordering of non-deterministic operations at the backup replicas. LLFT does not employ a majority-based, multiple-round consensus algorithm and, thus, it can operate in the common industrial case where there is a primary replica and only one backup replica. The LLFT system achieves low latency message delivery during normal operation and low latency reconfiguration and recovery when a fault occurs.

We present a lightweight Byzantine fault tolerance (BFT) algorithm, which can be used to render the coordination of web services business activities (WS-BA) more trustworthy. The lightweight design of the BFT algorithm is the result of a comprehensive study of the threats to the WS-BA coordination services and a careful analysis of the state model of WS-BA. The lightweight BFT algorithm uses source ordering, rather than total ordering, of incoming requests to achieve Byzantine fault tolerant, state-machine replication of the WS-BA coordination services. We have implemented the lightweight BFT algorithm, and incorporated it into the open-source Kandula framework, which implements the WS-BA specification with the WS-BA-I extension. Performance evaluation results obtained from the prototype implementation confirm the efficiency and effectiveness of our lightweight BFT algorithm, compared to traditional BFT techniques.

The Web Services Atomic Transactions (WS-AT) specification makes it possible for businesses to engage in standard distributed transaction processing over the Internet using Web Services technology. For such business applications, trustworthy coordination of WS-AT is crucial. In this paper, we explain how to render WS-AT coordination trustworthy by applying Byzantine Fault Tolerance (BFT) techniques. More specifically, we show how to protect the core services described in the WS-AT specification, namely, the Activation service, the Registration service, the Completion service and the Coordinator service, against Byzantine faults. The main contribution of this work is that it exploits the semantics of the WS-AT services to minimize the use of Byzantine Agreement (BA), instead of applying BFT techniques naively, which would be prohibitively expensive. We have incorporated our BFT protocols and mechanisms into an open-source framework that implements the WS-AT specification. The resulting BFT framework for WS-AT is useful for business applications that are based on WS-AT and that require a high degree of dependability, security, and trust.

The Low Latency Fault Tolerance (LLFT) middleware provides fault tolerance for distributed applications deployed within a cloud computing or data center environment, using the leader/follower replication approach. The LLFT middleware consists of a Low Latency Messaging Protocol, a Leader-Determined Membership Protocol, and a Virtual Determinizer Framework. The Messaging Protocol provides are liable, totally ordered message delivery service by employing a direct group-to-group multicast where the ordering is determined by the primary replica in the group. The Membership Protocol provides a fast reconfiguration and recovery service when a replica becomes faulty and when a replica joins or leaves a group. The Virtual Determinizer Framework captures ordering information at the primary replica and enforces the same ordering at the backup replicas for major sources of non-determinism. The LLFT middleware maintains strong replica consistency, offers application transparency, and achieves low end-to-end latency.

The use of good random numbers is essential to the integrity of many mission-critical systems. However, when such systems are replicated for Byzantine fault tolerance, a serious issue arises, i.e., how do we preserve the integrity of the systems while ensuring strong replica consistency? Despite the fact that there exists a large body of work on how to render replicas deterministic under the benign fault model, the solutions regarding the random number control are often overly simplistic without regard to the security requirement, and hence, they are not suitable for practical Byzantine fault tolerance. In this paper, we present a novel integrity-preserving replica coordination algorithm for Byzantine fault tolerant systems. The central idea behind this algorithm is that all random numbers to be used by the replicas are collectively determined, based on the contributions made by a quorum of replicas, at least one of which is correct. We have implemented the algorithm in Java and conducted extensive experiments, in both a LAN testbed and an emulated WAN environment. We show that our algorithm is particularly suited for Byzantine fault tolerant systems operating in the LAN environment, or where replicas are connected by high-speed low-latency networks.

With the advent of the new generation of Internet-based technology, in particular, web services, the automation of business activities that are distributed across multiple enterprises becomes possible. Business activities are different from traditional transactions in that they are typically asynchronous, loosely coupled, and long running. Therefore, extended transaction protocols are needed to coordinate business activities that span multiple enterprises. Existing extended transaction protocols typically rely on compensating transactions to handle exceptional conditions. In this paper, we identify a number of issues with compensation-based extended transaction protocols and describe a reservation-based extended transaction protocol that addresses those issues. Moreover, we define a set of properties, analogous to the ACID properties of traditional transactions that are more appropriate for business activities that span multiple enterprises. In addition, we compare our reservation protocol with other extended transaction protocols for coordinating business activities and present performance analyses and results.

In this paper, we present a Byzantine fault tolerant distributed commit protocol for transactions running over un- trusted networks. The traditional two-phase commit protocol is enhanced by replicating the coordinator and by running a Byzantine agreement algorithm among the coordinator replicas. Our protocol can tolerate Byzantine faults at the coordinator replicas and a subset of malicious faults at the participants. A decision certificate, which includes a set of registration records and a set of votes from participants, is used to facilitate the coordinator replicas to reach a Byzantine agreement on the outcome of each transaction. The certificate also limits the ways a faulty replica can use towards non-atomic termination of transactions, or semantically incorrect transaction outcomes.

All practical applications contain some degree of non- determinism. When such applications are replicated to achieve Byzantine fault tolerance (BFT), their nondeterministic operations must be controlled to ensure replica consistency. To the best of our knowledge, only the most simplistic types of replica nondeterminism have been dealt with. Furthermore, there lacks a systematic approach to handling common types of nondeterminism. In this paper, we propose a classification of common types of replica nondeterminism with respect to the requirement of achieving Byzantine fault tolerance, and describe the design and implementation of the core mechanisms necessary to handle such nondeterminism within a Byzantine fault tolerance framework.

In this paper, we present the design and implementation of a lightweight fault tolerance framework for Web services. With our framework, a Web service can be rendered fault tolerant by replicating it across several nodes. A consensusbased algorithm is used to ensure total ordering of the requests to the replicated Web service, and to ensure consistent membership view among the replicas. The framework is built by extending an open-source implementation of the WS-ReliableMessaging specification, and all reliable message exchanges in our framework conform to the specification. As such, our framework does not depend on any proprietary messaging and transport protocols, which is consistent with the Web services design principles. Our performance evaluation shows that our implementation is nearly optimal and the framework incurs only moderate runtime overhead.

Web services offer great promise for integrating and automating software applications within and between enterprises over the Internet. However, ensuring that Web services are dependable, and can satisfy their clients' requests when the clients need them is a real challenge because, typically, a business activity involves multiple Web services and a Web service involves multiple components, each of which must be dependable. In this paper, we describe fault tolerance techniques, including replication, checkpointing, and message logging, in addition to reliable messaging and transaction management for which Web services specifications exist. We discuss how those techniques can be applied to the components of the Web services involved in the business activities to render them dependable.

Traditional transaction semantics are not appropriate for business activities that involve long-running transactions in a loosely-coupled distributed environment, in particular, for Web services that operate between different enterprises over the Internet. In this paper we describe a novel reservation-based extended transaction protocol that can be used to coordinate such business activities. The protocol avoids the use of compensating transactions, which can result in undesirable effects. In our protocol, each task within a business activity is executed as two steps. The first step involves an explicit reservation of resources. The second step involves the confirmation or cancellation of the reservation. Each step is executed as a separate traditional short-running transaction. We show how our protocol can be implemented as a reservation protocol on top of the Web services transaction specification or, alternatively, as a coordination protocol on top of the Web services coordination specification.

In this paper, we describe a software infrastructure that unifies transactions and replication in three-tier architectures and provides data consistency and high availability for enterprise applications. The infrastructure uses transactions based on the CORBA object transaction service to protect the application data in databases on stable storage, using a roll-backward recovery strategy, and replication based on the fault tolerant CORBA standard to protect the middle-tier servers, using a roll-forward recovery strategy. The infrastructure replicates the middle-tier servers to protect the application business logic processing. In addition, it replicates the transaction coordinator, which renders the two-phase commit protocol nonblocking and, thus, avoids potentially long service disruptions caused by failure of the coordinator. The infrastructure handles the interactions between the replicated middle-tier servers and the database servers through replicated gateways that prevent duplicate requests from reaching the database servers. It implements automatic client-side failover mechanisms, which guarantee that clients know the outcome of the requests that they have made, and retries aborted transactions automatically on behalf of the clients.

Clock-related operations are one of the many sources of replica non-determinism and of replica inconsistency in fault-tolerant distributed systems. In passive replication, if the primary server crashes, the next clock value returned by the new primary server might have actually rolled back in time, which can lead to undesirable consequences for the replicated application. The same problem can happen for active replication where the result of the first replica to respond is taken as the next clock value. In this paper, we describe the design and implementation of a consistent time service for fault-tolerant distributed systems. The consistent time service introduces a group clock that is consistent across the replicas and that ensures the determinism of the replicas with respect to clock-related operations. The group clock is monotonically increasing, is transparent to the application and is fault-tolerant. The consistent time service guarantees the consistency of the group clock even when faults occur, when new replicas are added into the group and when failed replicas recover. 1

Critical components of a distributed system must be replicated to achieve high availability, and fault tolerance. Current fault tolerant CORBA infrastructures have concentrated on mechanisms for object replication and recovery, while rarely considering practical issues related to the context, i.e., the CORBA middleware within the process in which the object runs. Our study shows that to replicate and recover complex CORBA applications, the behavior of the process that hosts the CORBA objects, in particular the bootstrapping of an application, must be taken into account. In this paper, we discuss the challenges that arose when bootstrapping CORBA applications in some common scenarios, and we provide strategies to handle such difficulties so that CORBA applications can be rendered fault-tolerant.

In this paper we describe a software infrastructure that unifies replication and transaction processing in three-tier architectures and, thus, provides high availability and fault tolerance for enterprise applications. The infrastructure is based on the Fault Tolerant CORBA and CORBA Object Transaction Service standards, and works with commercial-off-the-shelf application servers and database systems. The infrastructure replicates the application servers to protect the business logic processing. In addition, it replicates the transaction coordinator which renders the two-phase commit protocol non-blocking and, thus, avoids potentially long service disruptions caused by coordinator failure. The infrastructure handles the interactions between the application servers and the database servers through replicated gateways that prevent duplicate requests from reaching the database servers. The infrastructure implements client-side automatic failover mechanisms, which guarantees that clients know the outcome of the requests that they have made. The infrastructure starts the transactions at the application servers, and retries aborted transactions, caused by process or communication failures, automatically on the behalf of the clients.

In this paper we describe an infrastructure that provides increased reliability for three-tier applications, transparently, using commercial off-the-shelf application servers and database systems. In this infrastructure the application servers are actively replicated to protect the business logic processing. Replicating the transaction coordinator renders the two-phase commit protocol non-blocking and, thus, avoids potentially long service disruptions caused by coordinator failure. A thin interpositioning library provides client-side automatic failover, so that clients know the outcome of their requests. The interaction between the application servers and the database servers is handled through replicated gateways that prevent duplicate requests from reaching the database servers. Aborted transactions, caused by process or communication faults, are automatically retried on the client's behalf.