Abstract As one of the most useful online processing techniques, the theta‐join operation has been utilized by many applications to fully excavate the relationships between data streams in various scenarios. As such, constant research efforts have been put to optimize its performance in the distributed environment, which is typically characterized by reducing the number of Cartesian products as much as possible. In this article, we design and implement a novel fast theta‐join algorithm, calledPrefap, by developing two distinct techniques—prefilteringandamalgamated partitioning—based on the state‐of‐the‐art FastThetaJoin algorithm to optimize the efficiency of the theta‐join operation. Firstly, we develop a prefiltering strategy before data streams are partitioned to reduce the amount of data to be involved and benefit a more fine‐grained partitioning. Secondly, to avoid the data streams being partitioned in a coarse‐grained isolated manner and improve the quality of the partition‐level filtering, we introduce an amalgamated partitioning mechanism that can amalgamate the partitioning boundaries of two data streams to assist a fine‐grained partitioning. With the integration of these two techniques into the existing FastThetaJoin algorithm, we design and implement a new framework to achieve a decreased number of Cartesian products and a higher theta‐join efficiency. By comparing with existing algorithms, FastThetaJoin in particular, we evaluate the performance ofPrefapon both synthetic and real data streams from two‐way to multiway theta‐join to demonstrate its superiority.

In sub-second stream computing, the answer to a complex query usually depends on operations of aggregation or join on streams, especially multi-way theta join. Some attribute keys are not distributed uniformly, which is called the data intrinsic skew problem, such as taxi car plate in GPS trajectories and transaction records, or stock code in stock quotes and investment portfolios etc. In this paper, we define the concept of key redundancy for single stream as the degree of data intrinsic skew, and joint key redundancy for multi-way streams. We present an execution model for multi-way stream theta joins with a fine-grained cost model to evaluate its performance. We propose a solution named Group Join (GroJoin) to make use of key redundancy during transmission and execution in a cluster. GroJoin is adaptive to data intrinsic skew in the way that it depends on the grouping condition we find out, i.e., the selectivity of theta join results should be smaller than 25%. Experiments are carried out by our MS-Generator to produce multi-way streams, and the simulation results show that GroJoin can decrease at most 45% transmission overheads with different key redundancies and value-key proportionality coefficients, and reduce at most 70% query delay with different key distributions. We further implement GroJoin in Multi-Way Stream Theta Join by Spark Streaming. The experimental results demonstrate that there are about 40%~50% join latency reduced after our optimization with a very small computation cost.

In this paper, we study the data caching problem in mobile cloud environment where multiple correlated data items could be packed and migrated to serve a predefined sequence of requests. By leveraging the spatial and temporal trajectory of requests, we propose a two-phase caching algorithm. We first investigate the correlation between data items to determine whether or not two data items could be packed to transfer, and then combine an existing dynamic programming (DP)-based algorithm and a greedy strategy to design a two-phase algorithm, named DP_Greedy, for effectively caching these shared data items to serve a predefined sequence of requests. Under homogeneous cost model, we prove the proposed algorithm is at most 2/α times worse than the optimal one in terms of the total service cost, where α is the defined discount factor, and also show that the algorithm can achieve this results within O(mn2) time and O(mn) space complexity for m caches to serve a n-length sequence. We evaluate our algorithm by effectively implementing it and comparing it with the non-packing case, the result show the proposed DP_Greedy algorithm not only presents excellent performances but is also more in line with the actual situation.

In this paper we consider a new caching model that enables data sharing for network services in a cost-effective way. The proposed caching algorithms are characterized by using monetary cost and access information to control the cache replacements, instead of exploiting capacity-oriented strategies as in traditional approaches. In particular, given a stream of requests to a shared data item with respect to a homogeneous cost model, we first propose a fast off-line algorithm using dynamic programming techniques, which can generate an optimal schedule within$O(mn)$time-space complexity by using cache, migration as well as replication to serve a$n$-length request sequence in a$m$-node network, substantially improving the previous results. Furthermore, we also study the online form of this problem, and present an 3-competitive online algorithm by leveraging an idea of anticipatory caching. The algorithm can serve an online request in constant time and is space efficient in$O(m)$as well, rendering it more practical in reality. We evaluate our algorithms, together with some variants, by conducting extensive simulation studies. Our results show that the optimal cost of the off-line algorithm is changed in a parabolic form as the ratio of caching cost to transfer cost is increased, and the online algorithm is less than 2 times worse in most cases than its optimal off-line counterpart.

In this paper we consider the data caching problem in next generation data services in the cloud, which is characterized by using monetary cost and access trajectory information to control cache replacements, instead of exploiting capacityoriented strategies as in traditional research. In particular, given a stream of requests to a shared data item with respect to a homogeneous cost model, we first propose a fast off-line algorithm using dynamic programming techniques. The proposed algorithm can generate optimal schedule within O(mn) timespace complexity to cache, migrate as well as replicate the shared data item to serve an n-length request sequence with minimum cost in a fully connected m-node network, substantially improving the previous results. Additionally, we also study this problem in its online form, and present a 3-competitive online algorithm by leveraging a speculative caching idea. The algorithm can serve an online request in constant time, and is space efficient in O(m) as well, rendering it to be more practical in reality. Our research complements the shortage of similar research in literature on this problem.

To predicate the future with high accuracy is a holy grail in financial market. However, the volatility of chaotic financial market challenges new technologies from computer science to economic science all the time. Recently, Recurrent Neural Network (RNN) plays a new role in financial market prediction. However, results from RNN are restricted by sample size of training datasets, and show predication accuracy can hardly be guaranteed in a long term. On the other hand, Representative Pattern Discovery (RPD) is an effective way in long-term prediction while it is ineffective in short-term prediction. In this paper, we define a representative pattern for time series, and propose a fusion financial prediction strategy based on RNN and RPD. We take the advantages of both RNN and RPD, in the way that the proposed strategy is stateful to keep the short-term trend and it rectifies the predication by a time-dependent incremental factor in a long-term way. Compared with RNN and pattern discovery respectively, our experimental results demonstrate that our proposed strategy performs much better than that of others. It can increase the prediction accuracy by 6% on the basis of RNN at most, but at a cost of higher Mean Squared Error.

Many exist localization algorithms are unbiased estimators. However, the estimation performance presents biased feature in the real location systems. On the other hand, many biased location estimators show advantages that unbiased estimators can not achieve, e.g., robust to the noise, more accurate estimation and low complexity. In this paper, we propose a biased localization estimator and a hybrid Kalman filtering algorithm. The proposed algorithm is robust to the complicated environment with high accuracy. Both theoretical analysis and experimental evaluation indicate that the proposed algorithm outperform the unbiased optimal estimation methods.

We present a network I/O allocation framework, called Raccoon, for workload-aware VM scheduling algorithm to facilitate hybrid I/O workloads in virtual environments. Raccoon combines the strengths of paravirtual I/O and SR-IOV techniques to not only minimize the network latency, but also optimize the bandwidth utilization for workload-aware VM scheduling. In Raccoon, a limited number of VFs in SR-IOV are granted to I/O-intensive VMs while the paravirtual Network Interface Cards (vNICs) are allocated to other non-I/O-intensive VMs as the default resources. With this design, Raccoon provides latency reduction and bandwidth guarantee under the premise that I/O-intensive VMs will always be granted the VFs to facilitate their I/O operations. The types of workloads in each VM are identified at runtime by modified XenMon. By leveraging the ACPI Hotplug technique, Raccoon can adaptively plugin and plugout the SR-IOV VFs upon the changes of VM requirements so that an efficient I/O workload-aware VM scheduling algorithm can be implemented based on the bonding driver technique. The experimental results reveal that Raccoon can combine the benefits of para-virtual I/O and SR-IOV techniques to improve the overall performance of virtualized platforms with VMs that have diverse I/O workloads.

Traffic lights in urban area can significantly influence the efficiency and effectiveness of transportation. The real-time scheduling information of traffic lights is fundamentally important for many intelligent transportation applications, such as shortest-time navigation and green driving advisory. However, existing traffic light scheduling identification systems either entail dedicated infrastructures or depend on specialized traffic traces, which hinders the popularity and real world deployment. Differently, we propose to identify real-time traffic light scheduling by analyzing taxi traces that are widely accessible from taxi companies. The key idea is to exploit the periodicity in traffic patterns, which is directly affected by traffic lights. We also develop advanced algorithms to identify red/green lights duration and signal change time. We evaluate our solution using over one billion taxi records from Shenzhen, China. The evaluation results validate the effectiveness of our system.

In an IMANET, mobile users access both text and media web contents on the Internet through gateway nodes, with web access patterns, i.e., the Zipf-like distribution or the Stretched Exponential distribution. To reduce data access delay from the Internet, we consider the cache placement problem in cooperative caching, which is that each mobile node selects a subset of web contents to cache cooperatively in its limited cache so that total access cost is minimized. It has been proved NP-hard. We propose a solution named Adaptive Allocation Cooperative Caching (AACC), which adaptively divides the cache space of each node into two components: altruistic and selfish, according to detected data access patterns. AACC aims to find the optimal cache allocation solution to allocate appropriate cache spaces for two components in order to minimize total access cost. Given the Zipf-like access distribution, we find a near-optimal allocation solution to the cache placement problem. Simulation results show that AACC achieves much better performance than the existing best cooperative caching strategy in IMANETs in terms of average query delay, caching overheads, and query success ratio. In particular, AACC reduces caching overheads by 40% in average.

Wireless ad hoc network is a promising networking technology to provide users with Internet access anywhere anytime. To cope with resource constraints of wireless ad hoc networks, data caching is widely used to efficiently reduce data access cost. In this paper, we propose an efficient data caching algorithm which makes use of the overhearing property of wireless communication to improve caching performance. Due to the broadcast nature of wireless links, a packet can be overheard by a node within the transmission range of the transmitter, even if the node is not the intended target. Our proposed algorithm explores the overheard information, including data request and data reply, to optimize cache placement and cache discovery. To the best of our knowledge, this is the first work that considers the overhearing property of wireless communications in data caching. The simulation results show that, compared with one representative algorithm and a naive overhearing algorithm, our proposed algorithm can significantly reduce both message cost and access delay.

Cooperative caching is an efficient way to improve the performance of data access in mobile wireless networks, by cache nodes cooperatively caching copies of data items in their limited storage. With more demands on sharing a video or other data, especially for mobile phone games in an Internet-based Mobile Ad Hoc Network, the relations among data items become much more important than before. However, most of the existing works do not consider inherent relations among these data items. In this paper, we present a novel solution, Gossip-based Cooperative Caching (GosCC), to address the cache placement problem, considering the sequential relation among data items. Each mobile node stores the IDs of data items cached locally and the ID of the data item in use into its progress report. Each mobile node also makes use of these progress reports to determine whether a data item should be cached locally. These progress reports are propagated within the network in a gossip-based way. To improve user experience, GosCC aims to provide users with an uninterrupted data access service. Simulation results show that GosCC achieves better performance than the best one of the existing solutions, Benefit-based Data Caching, in terms of average interruption intervals and average interruption times, while sacrificing message cost to a certain degree.

Data caching is one of the attractive techniques that can increase the efficiency of data access in wireless multi-hop ad hoc networks. However, it is still a challenging task to achieve an optimal trade-off between the total traffic cost and average access delay by properly selecting cache nodes, i.e. cache placement. In this paper, we address the problem of cache placement in wireless multi-hop ad hoc networks. We consider dynamic networks, in which there is a data source that stores one data item and other client nodes require accessing the data item. We define the cache placement problem on a dynamic network topology as Dynamic Cache Placement (DCP). Most of the existing cache placement algorithms use hop counts to measure the total cost of caching systems. Due to the impact of contentions in wireless networks, one hop delay is different from each other and it varies with the traffic load. Thus the previous algorithms cannot achieve their expected performance. We propose two heuristic cache placement algorithms, named Centralized Contention-Aware Caching Algorithm (CCCA) and Distributed Contention-Aware Caching Algorithm (DCCA), which detect the variation of contentions to evaluate the benefit of selecting a node as cache node. Simulation results show that the proposed algorithms achieve better performance than other alternative ones in terms of average query delay, caching overheads, and query success ratio.

Gossiping has been widely used for disseminating data in large scale networks. Existing works have mainly focused on the design of gossip-based protocols but few have been reported on developing models for analyzing the fault tolerance property of these protocols. In this paper, we propose a general gossiping algorithm and develop a mathematical model based on generalized random graphs for evaluating the reliability of gossiping, i.e., to what extent gossip-based protocols can tolerate node failures, yet guarantee the specified message delivery. We analytically derive the maximum ratio of failed nodes that can be tolerated without reducing the required degree of reliability. We also investigate the impact of the parameters, namely the fanout distribution and the nonfailed member ratio, on the protocol reliability. Simulations have been carried out to validate the effectiveness of our analytic model in terms of the reliability of gossiping and the success of gossiping. The results obtained can be used to guide the design of fault tolerant gossip-based protocols.

This paper presents the design and evaluation of a highly efficient on-demand multicast routing protocol for mobile ad hoc networks (MANETs). The protocol, called Geography-aided Multicast Zone Routing Protocol (GMZRP), eliminates as much as possible duplicate route queries by using a simple yet eff