Data analytics systems commonly utilize in-memory query processing techniques to achieve better throughput and lower latency. Modern computers increasingly rely on Non-Uniform Memory Access (NUMA) architectures to achieve scalability. A key drawback of NUMA architectures is that many existing software solutions are not aware of the underlying NUMA topology and thus do not take full advantage of the hardware. Modern operating systems are designed to provide basic support for NUMA systems. However, default system configurations are typically sub-optimal for large data analytics applications. Additionally, rewriting the application from the ground up is not always feasible.In this work, we evaluate a variety of strategies that aim to accelerate memory-intensive data analytics workloads on NUMA systems. Our findings indicate that the operating system default configurations can be detrimental to query performance. We analyze the impact of different memory allocators, memory placement strategies, thread placement, and kernel-level load balancing and memory management mechanisms. With extensive experimental evaluation, we demonstrate that the methodical application of these techniques can be used to obtain significant speedups in four commonplace in-memory query processing tasks, on three different hardware architectures. Furthermore, we show that these strategies can improve the performance of five popular database systems running a TPC-H workload. Lastly, we summarize our findings in a decision flowchart for practitioners.

Location-based services (LBS) are a widely adopted technology that produces large volumes of spatio-temporal data at high velocity. Spatial data is also being generated from many other geo-spatial applications. To address the challenge of data volume, a number of big spatial data management systems have emerged that are based on the MapReduce paradigm. Recent projects have developed spatial data systems using Spark's distributed in-memory architecture. These projects, which include GeoSpark, SpatialSpark, and LocationSpark, do not support the high update rates required by LBS applications. Alternatively, systems such as MD-HBase support data updates, but are hindered by the performance characteristics of HBase, which is a disk-oriented framework. We present DISTIL+, a distributed spatio-temporal data processing system designed for high velocity location data. Our system achieves high update throughput and low query latency by leveraging the APGAS (Asynchronous Partitioned Global Address Space) architecture to build a multi-level distributed in-memory index. We present extensive experimental evaluation of our system, comparing several indexing and data placement schemes, as well as competing systems. Our results show that DISTIL+ excels at supporting high throughput location updates, and low latency spatio-temporal range queries and kNN queries, while offering better performance than existing approaches.

Location-based services (LBS) have become an ubiquitous technology and spatio-temporal data generated by LBS is characterized by high volume and velocity. In recent times several projects, such as GeoSpark, SpatialSpark and LocationSpark, have focused on developing spatial data systems that take advantage of the distributed in-memory data processing capability of Spark. However, most of these systems assume immutable spatial data, and they do not support high throughput location data updates that are common in LBS. On the other hand, a few HBase-based systems, such as MD-HBase, have been proposed that support data updates. However, these systems do not take advantage of any distributed in-memory query processing frameworks.

Main memory hash joins are an important category of in-memory joins. However, the performance of these joins can be hindered by dataset skew, shuffling, and load balancing. We conducted a comprehensive study on the effects of dataset skew on four hash join algorithms. We show that hash joins are acutely affected by dataset skew, and the performance gets worse with shuffled data. To address these issues, we propose non-partitioning hash joins using two different hash tables. First, we use a separate chaining hash table that is based on an existing implementation that we have modified. This version outperforms the original implementation on skewed datasets by up to three orders of magnitude. Second, we propose a novel hash table for hash joins, called Maple hash table. We demonstrate that this hash table is better suited to skewed and/or shuffled datasets. Moreover, this approach further improves performance by up to 17.3×.

Automated matchmaking is increasingly used in e-marketplaces. A participant is presented with a ranked list of coun- terpart profiles of other participants. A participant profile consists of a set of hard (inflexible) and soft (flexible) con- straints. For two participant profiles, corresponding pairs of constraints are compared. We propose compromise match- ing wherein one or both constraints of a pair are soft and their values do not match exactly. We demonstrate com- promise matching with a home rental use case with data obtained from an existing e-marketplace.

Fuzzy C-means (FCM) and Rough K-means (RKM) algorithms are two popular soft clustering algorithms that allow for overlapping clusters. The overlapping clusters can be useful in applications where restrictions imposed by crisp clustering that force assignment of every object to a unique cluster may not be practical. Likewise RKM and FCM, interval set representation of clusters would also generate overlapping clusters. We present and discuss the interval set K-means algorithm (IKM). This paper applies RKM, FCM and IKM algorithms for clustering web visits to an educational site. The experimental comparison highlights various features of these three soft computing algorithms.

The success of matchmaking systems largely depends on how effectively product/service descriptions (profiles) of participants are modelled. We formalize the multifaceted expectations and interests of participants as 'constraints' in those profiles. We identify and explicitly define the relevant types of constraints. We propose a new knowledge representation (KR) model for Web-based matchmaking systems that can represent these constraints. We present a system that implements the proposed KR model, exemplifying its features and evaluating its performance.

Parallelization of sequential programs is often daunting because of the substantial development cost involved. Previous solutions have not always been successful, partly because many try to address all types of applications. We propose a platform for parallelization of a class of applications that have similar computational structure, namely graph-structured iterative applications. iC2mpi is a unique proof-of-concept prototype platform that provides relatively easy parallelization of existing sequential programs and facilitates experimentation with static partitioning and dynamic load balancing schemes. We demonstrate with various generic application graph topologies that our platform can produce good performance with very little effort. The iC2mpi platform has a good potential for further performance improvements and for extensions to related classes of application domains.

Uniting ontologies and rules has become a central topic in the Semantic Web. Bridging the discrepancy between these two knowledge representations, this paper introduces DatalogDL as a family of hybrid languages, where Datalog rules are parameterized by various DL (description logic) languages ranging from to . Making DatalogDL a decidable system with complexity of EXPTIME, we propose independent properties in the DL body as the restriction to hybrid rules, and weaken the safeness condition to balance the trade‐off between expressivity and reasoning power. Building on existing well‐developed techniques, we present a principled approach to enrich (RuleML) rules with information from (OWL) ontologies, and develop a prototype system combining a rule engine (OO jDREW) with a DL reasoner (RACER).

We have proposed earlier the incremental internode communication method to reduce the communication cost as well as the time of the learning process in artificial neural networks (ANNs). In this paper, the limited precision incremental communication method is applied to a class of recurrent neural networks, the adaptive resonance theory 2 (ART2) networks. Simulation studies are carried out to examine the effects of the incremental communication method on the convergence behavior of ART2 networks. We have found that, 7-13-b precision is sufficient to obtain almost the same results as those with full (32-b) precision conventional communication. A theoretical error analysis is also carried out to analyze the effects of the limited precision incremental communication. The simulation and analytical results show that the limited precision errors are bounded and do not seriously degrade the convergence of ART2 networks. Therefore, the incremental communication can be incorporated in parallel and special-purpose very large scale integration (VLSI) implementations of the ART2 networks.

A tree similarity algorithm for match‐making of agents in e‐Business environments is presented. Product/service descriptions of seller and buyer agents are represented asnode‐labeled, arc‐labeled, arc‐weightedtrees. A similarity algorithm for such trees is developed as the basis for semantic match‐making in a virtual marketplace. The trees are exchanged using an XML serialization in Object‐Oriented RuleML. Correspondingly, we use the declarative language Relfun to implement the similarity algorithm as a parameterized, recursive functional program. Three main recursive functions perform a top‐down traversal of trees and the bottom‐up computation of similarity. Results from our experiments aiming to match buyers and sellers are found to be effective and promising for e‐Business/e‐Learning environments. The algorithm can be applied in all environments where weighted trees are used.

Artificial neural networks (ANNs) involve a large amount of internode communications. To reduce the communication cost as well as the time of learning process in ANNs, we earlier proposed (1995) an incremental internode communication method. In the incremental communication method, instead of communicating the full magnitude of the output value of a node, only the increment or decrement to its previous value is sent to a communication link. In this paper, the effects of the limited precision incremental communication method on the convergence behavior and performance of multilayer neural networks are investigated. The nonlinear aspects of representing the incremental values with reduced (limited) precision for the commonly used error backpropagation training algorithm are analyzed. It is shown that the nonlinear effect of small perturbations in the input(s)/output of a node does not cause instability. The analysis is supported by simulation studies of two problems. The simulation results demonstrate that the limited precision errors are bounded and do not seriously affect the convergence of multilayer neural networks.

A new method of inter-neuron communication called incremental communication is presented. In the incremental communication method, instead of communicating the whole value of a variable, only the increment or decrement of its previous value is sent on a communication link. The incremental value may be either a fixed-point or a floating-point value. Multilayer feedforward network architecture is used to illustrate the effectiveness of the proposed communication scheme. The method is applied to three different learning problems and the effect of the precision of incremental input-output values of the neurons on the convergence behavior is examined. It is shown through simulation that for some problems even four-bit precision in fixed- and/or floating-point representations is sufficient for the network to converge. With 8-12 bit precisions almost the same results are obtained as that with the conventional communication using 32-bit precision. The proposed method of communication can lead to significant savings in the intercommunication cost for implementations of artificial neural networks on parallel computers as well as the interconnection cost of direct hardware realizations. The method can be incorporated into most of the current learning algorithms in which inter-neuron communications are required. Moreover, it can be used along with the other limited precision strategies for representation of variables suggested in literature.