Resource disaggregation (RD) is an emerging paradigm for data center computing whereby resource-optimized servers are employed to minimize resource fragmentation and improve resource utilization. Apache Spark deployed under the RD paradigm employs a cluster of compute-optimized servers to run executors and a cluster of storage-optimized servers to host the data on HDFS. However, the network transfer from storage to compute cluster becomes a severe bottleneck for big data processing. Near-data processing (NDP) is a concept that aims to alleviate network load in such cases by offloading (or "pushing down") some of the compute tasks to the storage cluster. Employing NDP for Spark under the RD paradigm is challenging because storage-optimized servers have limited computational resources and cannot host the entire Spark processing stack. Further, even if such a lightweight stack could be developed and deployed on the storage cluster, it is not entirely obvious which Spark queries would benefit from pushdown, and which tasks of a given query should be pushed down to storage.This paper presents the design and implementation of a near-data processing system for Spark, SparkNDP, that aims to address the aforementioned challenges. SparkNDP works by implementing novel NDP Spark capabilities on the storage cluster using a lightweight library of SQL operators and then developing an analytical model to help determine which Spark tasks should be pushed down to storage based on the current network and system state. Simulation and prototype implementation results show that SparkNDP can help reduce Spark query execution times when compared to both the default approach of not pushing down any tasks to storage and the outright NDP approach of pushing all tasks to storage.

Presents the farewell editorial from the editor for this issue of the publication.

Healthcare enterprises are starting to adopt cloud computing due to its numerous advantages over traditional infrastructures. This has become a necessity because of the increased volume, velocity and variety of healthcare data, and the need to facilitate data correlation and large-scale analysis. Cloud computing infrastructures have the power to offer continuous acquisition of data from multiple heterogeneous sources, efficient data integration, and big data analysis. At the same time, security, availability, and disaster recovery are critical factors aiding towards the adoption of cloud computing. However, the migration of healthcare workloads to cloud is not straightforward due to the vagueness in healthcare data standards, heterogeneity and sensitive nature of healthcare data, and many regulations that govern its usage. This paper highlights the need for providing healthcare data acquisition using cloud infrastructures and presents the challenges, requirements, use-cases, and best practices for building a state-of-the-art healthcare data ingestion service on cloud.

Reports on the state of the IEEE Transactions on Cloud Computing for 2018. The publication has been indexed in Clarivate Analytics’ Web of Science under the Science Citation Index (SCI), and has received its initial impact factor assessment. According to the 2018 Journal Citation Reports (based on 2017 data), the TCC has a journal impact factor of 7.928 and is ranked third among all journals around the world in the category of “Computer Science - Information Systems”. The impressive impact factor also places the TCC in the top 5 percent of the approximately 200 IEEE publications across Computer Sciences, Electrical and Electronic Engineering, and Telecommunications.

Presents the editor's review of the state of the Journal for this issue of the publication.

Data mapping among different data standards in health institutes is often a necessity when data exchanges occur among different institutes. However, no matter rule-based approaches or traditional machine learning methods, none of these methods have achieved satisfactory results yet. In this work, we propose a deep learning method, mixture feature embedding convolutional neural network (MfeCNN), to convert the data mapping to a multiple classification problem. Multi-modal features were extracted from different semantic space with a medical NLP package and powerful feature embeddings were generated by MfeCNN. Classes as many as ten were classified simultaneously by a fully-connected soft-max layer based on multi-view embedding. Experimental results show that our proposed MfeCNN achieved best results than traditional state-of-the-art machine learning models and also much better results than the convolutional neural network of only using bag-of-words as inputs.

Consumer ratings are widely used in online marketplaces-helping vendors in assessing the quality of offerings and consumers in discovery and purchase decisions. To build trust in a marketplace, which has a direct impact on sales, an accurate assessment of ratings is essential in determining the quality of offerings. This paper proposes novel extensions to consumer Rating as a Service (RaaS)-a rating management service providing consumer rating functionality to a marketplace using hierarchical aggregation, which is a rating aggregation mechanism using hierarchical relationships of components to evaluate composite offerings. Contributions include the optimization of RaaS design for Web-scale, the integration of consumer credibility in hierarchical aggregation, and the application of hierarchical aggregation to existing independent atomic offerings. Various experiments are conducted to demonstrate the practicality of RaaS and correctness of hierarchical aggregation using real ratings from Amazon.com.

The pervasive computing discipline has evolved substantially since its inception. A lot of progress has been made in scaling pervasive computing, which has led to new, societal-scale capabilities such as mobile crowdsensing and the Internet of Things. Pervasive computing applications have also expanded from smart appliances and smart environments in the early days to systems of much broader scope such as smart cities and smart transportation. Going forward, an important direction is to enable a significantly heightened level of smartness in pervasive computing systems, by leveraging developments in big data and cognitive computing. Specifically, with its exponentially increasing volume, velocity and variety, big data is hailed as the world's new natural resource, and is driving fundamental changes in technology, business, and society. The biggest value of big data lies in the deep, actionable insights that can be derived from integrating all sources and modalities of data, across pervasive sensors and enterprise information systems. Such insights can then be exploited for business innovations and competitive advantages. Cognitive computing is a key enabling technology for turning big data into insights. Different from traditional programmable systems, cognitive systems are able to understand human knowledge, reason with a purpose, and learn and improve over time. Pervasive applications that integrate big data across the board and are cognitive are thus more insightful and can offer an unprecedented level of intelligence. In this talk, I will discuss the implications of big data and cognitive computing on pervasive computing. I will draw upon our experience at IBM Watson Health, and discuss how big data and cognitive computing can come together with pervasive computing to enable innovative health solutions that address many clinical, societal, and economic issues. I will present use cases, highlight the challenges, describe our approaches, and relate to client experiences.

Reports on the scope, contents, and state of the IEEE Transactions on Cloud Computing.

In many mobile sensing applications, devices need to discover new neighbors and maintain the rendezvous with known neighbors continuously. Due to the limited energy supply, these devices have to duty cycle their radios to conserve the energy and bandwidth, making neighbor discovery and rendezvous maintenance even more challenging. To date, the main mechanism for device discover and rendezvous maintenance in existing solutions is pairwise, direct one-hop communication. We argue that such pairwise direct communication is sufficient but not necessary: there exist unnecessary active slots that can be eliminated, without affecting discovery and rendezvous. In this work, we propose a novel concept ofextended quorum system, which leveragesindirectdiscovery to further conserve energy. Specifically, we usequorum graphto capture all possible information flow paths where knowledge about known-neighbors can propagate among devices. By eliminating redundant paths, we can reduce the number of active slots significantly. Since a quorum graph can characterize arbitrary active schedules of mobile devices, our work can be broadly used to improve many existing quorum-based discovery and rendezvous solutions. We comprehensively evaluate$EQS$in three different scales of networks, and the results show that$EQS$reduces as much as 55 percent energy consumption with a maximal 5 percent increase in latency for existing solutions. To test the real-world values of$EQS$, we further propose a taxicab dispatching application called$EQS$-dispatch to navigate taxicab drivers to the area with less competition based on the discovery results of nearby taxicabs.

Service-oriented Architecture (SOA) comprises a number of loosely-coupled independent services, which collaborate, interact and share data to accomplish incoming requests. A service invocation can involve multiple services, where each service accesses, processes and shares the client's data. These interactions may share data with unauthorized services and violate client's privacy. The client has no means of identifying if a violation occurred because it has no control over the service invocations beyond its trust domain. Such interactions introduce new security challenges which are not present in traditional systems. This paper proposes a data-centric approach for privacy preserving access control in SOA. Benefits of the proposed approach include the ability to dynamically define access polices by the clients and control data access at the time of each service interaction. A realistic healthcare scenario is used to evaluate the implementation of the proposed solution which validates its viability.

Presents the introductory editorial for this issue of the publication.

As a supporting primitive of many mobile applications, neighbor discovery identifies nearby devices so that they can exchange information and collaborate in a peer-to-peer manner. To date, discovery schemes trade a long latency for energy efficiency and require a collaborative duty cycle pattern, and thus they are not suitable for interactive mobile applications where a user is unable to configure others’ devices. In this article, we propose Acc , which serves as an on-demand generic discovery accelerating middleware for many deterministic neighbor discovery schemes. Acc leverages the discovery capabilities of neighbor devices, supporting both direct and indirect neighbor discoveries. Further, we present a proactive online rendezvous maintenance mechanism, which is used to reduce delays for the detection of leaving of neighbors. Our evaluations show that Acc -assisted discovery schemes reduce latency by up to 51.8% compared to schemes consuming the same amount of energy. More importantly, to prove the real-world value of Acc , we further present and evaluate a Crowd-Alert application where Acc is employed by taxi drivers to accelerate selection of a direction with fewer competing taxis and more potential passengers, based on a 280GB dataset of more than 14,000 taxis in Shenzhen, the most crowded city in China.

In many mobile sensing applications devices need to discover new neighbors and maintain the rendezvous with known neighbors continuously. Due to the limited energy supply, these devices have to cycle their radios to conserve energy, making neighbor discovery and rendezvous maintenance even more challenging. To date, the main mechanism for device discover and rendezvous maintenance in existing solutions is pair wise, direct one-hop communication. We argue that such pair wise direct communication is sufficient but not necessary: there exist unnecessary active slots that can be eliminated, without affecting discovery and rendezvous. In this work, we propose a novel concept of extended quorum system, which leverages indirect discovery to further conserve energy. Specifically, we use quorum graph to capture all possible information flow paths where knowledge about known-neighbors can propagate among devices. By eliminating redundant paths, we can reduce the number of active slots significantly. Since a quorum graph can characterize arbitrary active schedules of mobile devices, our work can be broadly used to improve many existing quorum based discovery and rendezvous solutions. The simulation and test bed experimental results show that our solution can reduce as much as 55% energy consumption with a maximal 5% increase in latency for existing solutions.

Cloud computing centers face the key challenge of provisioning diverse virtual machine instances in an elastic and scalable manner. To address this challenge, we have performed an analysis of VM instance traces collected at six production data centers during four months. One key finding is that the number of instances created from the same VM image is relatively small at a given time and thus conventional file-based p2p sharing approaches may not be effective. Based on the understanding that different VM image files often have many common chunks of data, we propose a chunk-level Virtual machine image Distribution Network (VDN). Our distribution scheme takes advantage of the hierarchical network topology of data centers to reduce the VM instance provisioning time and also to minimize the overhead of maintaining chunk location information. Evaluation shows that VDN achieves as much as 30-80× speed up for large VM images under heavy traffic.

As a supporting primitive of many mobile device applications, neighbor discovery identifies nearby devices so that they can exchange information and collaborate in a peer-to-peer manner. To date, discovery schemes trade a long latency for energy efficiency and require a collaborative duty cycle pattern, and thus they are not suitable for interactive mobile applications where a user is unable to configure others' devices. In this paper, we propose Acc, which serves as an on-demand generic discovery accelerating middleware for many existing neighbor discovery schemes. Acc leverages the discovery capabilities of neighbor devices, supporting both direct and indirect neighbor discoveries. Our evaluations show that Acc-assisted discovery schemes reduce latency by a maximum of 51.8%, compared with the schemes consuming the same amount of energy. We further present and evaluate a Crowd-Alert application where Acc can be employed by taxi drivers to accelerate selection of a direction with fewer competing taxis and more potential passengers, based on a 10 GB dataset of more than 15,000 taxis in a metropolitan area.

The number of real-world applications that require QoS guarantees is constantly increasing and they often follow the publish/subscribe (pub/sub)messaging paradigm, which provides loosely coupled many-to-many communication. Many QoS-aware systems use overlay networks as they allow flexible routing. To provide QoS-aware pub/sub messaging in overlay networks, the messaging system should be adaptive to the changes in network conditions (such as delay and failures). However, many pub/sub systems depend on a flexed routing topology and it is costly to rebuild this topology in case of failures. This study seeks to address this challenge with Delay-Cognizant Reliable Delivery (DCRD), a novel and delay-aware dynamic routing algorithm to provide reliable message delivery for pub/sub overlay networks. For reliable message delivery, DCRD no longer uses a flxed routing topology. Instead, it dynamically switches among different links to bypass link failures and increase the chance to meet QoS requirement. Each node tries different neighboring nodes in an order that is mathematically proven to minimize the expected delay of packet delivery. With all possible neighboring nodes sorted this way, DCRD guarantees that packets are delivered as long as there exists a path between the publisher and subscriber and that the expected delay is minimized. DCRD is extensively evaluated in simulation with comparison to existing tree-based routing approaches as well as a multi path approach using different network topologies, delay constraints, and loss probabilities. Simulation results show that DCRD performs better than all the baselines, providing reliable message delivery and satisfying the delay requirement for more than 98% of messages when the link failure probability is 4% or less.

The rapid growth of sense-and-respond applications and the emerging cloud computing model present a new challenge: providing publish/subscribe as a scalable and elastic cloud service. This paper presents the Blue Dove attribute based publish/subscribe service that seeks to address such a challenge. Blue Dove uses a gossip-based one-hop overlay to organize servers into a scalable cluster. It proactively exploits skewness in data distribution to achieve high performance. By assigning each subscription to multiple servers through a multidimensional subscription space partitioning technique, it provides multiple candidate servers for each publication message. A message can be matched on any of its candidate servers with one hop forwarding. The performance-aware forwarding in Blue Dove ensures that the message is sent to the least loaded candidate server for processing, leading to low latency and high throughput. The evaluation shows that Blue Dove has a linear capacity increase as the system scales up, adapts to sudden workload changes within tens of seconds, and achieves multifold higher throughput than the techniques used in the existing enterprise and peer-to-peer pub/sub systems.

Queuing is commonly used to connect loosely coupled components to form large-scale, highly-distributed, and fault-tolerant applications. As cloud computing continues to gain popularity, a number of vendors have started offering cloud-hosted, multi-tenant queuing service. They provide high availability at the cost of reduced consistency. Although they offer at-least-once delivery guarantee, that is, no message loss, they do not make any effort in maintaining FIFO order, which is an important aspect of the queuing semantics. Thus they are not adequate for some applications. This paper presents the design and implementation of a scalable cloud-based queuing service, called Blue Dove Queuing Service (BDQS). It provides improved queuing consistency - at-least-once and best-effort in-order message delivery - while preserving high availability and reliability. It also offers clients a flexible trade-off between duplication and message order. Comprehensive evaluation is carried out on an Infrastructure-as-a-Service cloud computing platform with up to 70 server nodes and 1000 queues. It shows that BDQS achieves excellent performance scalability. Meanwhile, it offers an order-of-magnitude improvement in out-of-order measurement compared to existing no-order systems. Results also indicate that BDQS is highly reliable and available.

Stream processing is widely used by today's applications such as financial data analysis and disaster response. In distributed stream processing systems, machine fail-stop events are handled by either active standby or passive standby. However, existing high availability (HA) schemes have not sufficiently addressed the situation when a machine becomes temporarily unavailable due to data rate spikes, intensive analysis or job sharing, which happens frequently but lasts for short time. It is not clear how well active and passive standby fare against such transient unavailability. In this paper, we first critically examine the suitability of active and passive standby against transient unavailability in a real testbed environment. We find that both approaches have advantages and drawbacks, but neither is ideal to provide fast recovery at low overhead as required to handle transient unavailability. Based on the insights gained, we propose a novel hybrid HA method that switches between active and passive standby modes depending on the occurrence of failure events. It presents a desirable tradeoff that is different from existing HA approaches: low overhead during normal conditions and fast recovery upon transient or permanent failure events. We have implemented our hybrid method and compared it with existing HA designs with comprehensive evaluation. The results show that our hybrid method can reduce two-thirds of the recovery time compared to passive standby and 80% message overhead compared to active standby, allowing applications to enjoy uninterrupted processing without paying a high premium.

Multipath streaming protocols have recently attracted much attention because they provide an effective means to provide high-quality streaming over the Internet. However, many existing schemes require a long start-up delay and thus are not suitable for interactive applications such as video conferencing and tele-presence. In this paper, we focus on real-time live streaming applications with stringent end-to-end latency requirement, say several hundreds of milliseconds. To address these challenges, we take a joint multipath and FEC approach that intelligently splits the FEC-encoded stream among multiple available paths. We develop an analytical model and use asymptotic analysis to derive closed-form, optimal load splitting solutions, which are surprisingly simple yet insightful. To our best knowledge, this is the first work that provides such closed-form optimal solutions. Based on the analytical insights, we have designed and implemented a novel encoded multipath streaming (EMS) scheme for real-time live streaming. EMS strives to continuously satisfy the application's QoS requirements by dynamically adjusting the load splitting decisions and the FEC settings. Our simulation results have shown that EMS can not only outperform the existing multipath streaming schemes, but also adapt to the dynamic loss and delay characteristics of the network with minimal overhead.

Compared with the traditional client/server streaming model, peer-assisted video streaming has been shown to provide better scalability with lower infrastructure cost. In this paper, we describe how peer-assisted video streaming can be implemented through real-time service oriented architecture. Our architecture, dubbed PACTS, is composed of 5 distinct service components: directory service, content service, peer download service, peer upload service and rate composition service. PACTS is designed to meet the specific QoS requirements of different users, entered through a simple Web interface. It also provides a distributed implementation of the content and peer directory services for effectively handling flash crowd situations, when a large number of users join during a very short period of time. We discuss the functionalities of the main PACTS services and specify the workflow of how these services work together to deliver real-time streaming services leveraging the benefits of peer to peer technology. By organizing elements of traditional video streaming and peer to peer computing into loosely-coupled composable middleware services and distributing them among participating entities, PACTS enables high-quality low- cost video streaming at a large scale and in real time.

Enabling people-to-people interaction across heterogeneous communication end-points enhances user experience and fosters people collaboration. This paper presents the design and implementation of a unified communication system, dubbed Mercury, that allows a user to interact with others using the most convenient device at the time. Mercury supports both two-way conversation and one-way messaging. It leverages the Session Initiation Protocol to manage communication sessions and exploits dynamic user context to proactively route and migrate calls. It allows for subscription to other users' unified reachability status and provides a soft ring feature via universal notification. Mercury has an extensible architecture that allows new device types to be easily incorporated into the system. Our prototype implementation integrates a variety of devices: telephones, Sametime instant messaging clients, email, and pagers.

People are an important part of many business processes. Current workflow-based implementations of business processes constrain users to the desktop environment; require them to periodically check for pending tasks; and do not support direct or synchronous people-to-people interaction. On the other hand, the wide spectrum of people collaboration tools ranging from telephones to instant messaging and to email have no provision for structured collaboration and are separate from business processes. We have designed and implemented a system, dubbed PerCollab that integrates workflow and collaboration technologies. It allows people to participate in business processes from anywhere using a traditional collaboration mechanism. It proactively engages users in business processes by pushing interaction to a convenient device of the users. PerCollab uses an extended BPEL to formally define business processes with human partners and exploits dynamic user context to solve the personal mobility problem. Our prototype implementation integrates a variety of collaboration tools: instant messaging, email and e-meeting.

We propose a new algorithm, called Stripe-join, for performing a join given a join index. Stripe-join is inspired by an algorithm called ‘Jive-join’ developed by Li and Ross. Stripe-join makes a single sequential pass through each input relation, in addition to one pass through the join index and two passes through a set of temporary files that contain tuple identifiers but no input tuples. Stripe-join performs this efficiently even when the input relations are much larger than main memory, as long as the number of blocks in main memory is of the order of the square root of the number of blocks in the participating relations. Stripe-join is particularly efficient for self-joins. To our knowledge, Stripe-join is the first algorithm that, given a join index and a relation significantly larger than main memory, can perform a self-join with just a single pass over the input relation and without storing input tuples in intermediate files. Almost all the I/O is sequential, thus minimizing the impact of seek and rotational latency. The algorithm is resistant to data skew. It can also join multiple relations while still making only a single pass over each input relation. Using a detailed cost model, Stripe-join is analyzed and compared with competing algorithms. For large input relations, Stripe-join performs significantly better than Valduriez's algorithm and hash join algorithms. We demonstrate circumstances under which Stripe-join performs significantly better than Jive-join. Unlike Jive-join, Stripe-join makes no assumptions about the order of the join index.

We propose a new algorithm, called Stripe-join, for performing a join given a join index. Stripe-join is inspired by an algorithm called "Jive-join" developed by Li and Ross. Stripe-join makes a single sequential pass through each input relation, in addition to one pass through the join index and two passes through a set of temporary files that contain tuple identifiers but no input tuples. Stripe-join performs this efficiently even when the input relations are much larger than main memory, as long as the number of blocks in main memory is of the order of the square root of the number of blocks in the participating relations. Stripe-join is particularly efficient for self-joins. To our knowledge, Stripe-join is the first algorithm that, given a join index and a relation significantly larger than main memory, can perform a selfjoin with just a single pass over the input relation and without storing input tuples in intermediate files. Almost all the I/O is sequential, thus minimizing the...

Mobile computing adds a new wrinkle to the ageold problem of caching. Today's wireless links are both slow and expensive, and are not always available to a user. Therefore, when a mobile user is disconnected, a cache miss means (at best) a substantial cost in time and money, or (at worst) a complete halt to work if critical information has not been cached. Existing solutions to this problem rely on some combination of explicit hoard profiles and spying on a user's file accesses.