Mammoth transactions, which involve long-running operations that access many items, are common in graph workloads. Graph analytics tasks, including pattern matching and graph algorithms, can generate large read-write operations that impact significant portions of data, which makes their execution challenging under strict isolation guarantees. Consequently, we face an apparent trade-off between ensuring high isolation and achieving high performance, forcing users to choose between the two. In this work, we present TuskFlow, an experimental graph database based on Neo4j, designed to efficiently handle mammoth transactions on graphs (the technique is applicable to other models such as relational) while maintaining existing transactional semantics. TuskFlow employs a deterministic protocol that safely reorders regular transactions around mammoths within an epoch. Our protocol supports parallel mammoth execution inspired by graph-parallel algorithms. To minimize conflicts with regular transactions, TuskFlow introduces query- and workload-aware optimizations, including graph entity tagging and partitioning. Our experiments demonstrate that, unlike traditional protocols like two-phase locking or MVCC, TuskFlow avoids blocking write transactions and improves tail latency by up to 45×.

This paper argues for better concurrency control to support mammoth transactions , which read and write to many items. While these requests are prevalent on graph data, few systems support them efficiently. Currently, developers must make the uncomfortable choice between accepting dismal performance or abandoning transactional semantics. Applications deserve better: we believe that inherent graph properties provide a path forward to designing efficient concurrency control that preserves strong isolation.

Graph partitioning has long been seen as a viable approach to addressing Graph DBMS scalability. A partitioning, however, may introduce extra query processing latency unless it is sensitive to a specific query workload, and optimised to minimise inter-partition traversals for that workload. Additionally, it should also be possible to incrementally adjust the partitioning in reaction to changes in the graph topology, the query workload, or both. Because of their complexity, current partitioning algorithms fall short of one or both of these requirements, as they are designed for offline use and as one-off operations. The TAPER system aims to address both requirements, whilst leveraging existing partitioning algorithms. TAPER takes any given initial partitioning as a starting point, and iteratively adjusts it by swapping chosen vertices across partitions, heuristically reducing the probability of inter-partition traversals for a given path queries workload. Iterations are inexpensive thanks to time and space optimisations in the underlying support data structures. We evaluate TAPER on two different large test graphs and over realistic query workloads. Our results indicate that, given a hash-based partitioning, TAPER reduces the number of inter-partition traversals by $$\sim $$ 80%; given an unweighted Metis partitioning, by $$\sim $$ 30%. These reductions are achieved within eight iterations and with the additional advantage of being workload-aware and usable online.

This paper describes the development and delivery of a course, in close collaboration with industry, over a ten-year period. We describe the details of this collaboration, which aims to equip students with collaborative software development experience, incorporating open source, Enterprise Middleware technologies and industry best practices. In presenting our case study we share valuable insights, into the challenges and opportunities, for all parties involved. We present the reflections of all stakeholders, including; the course leader, the industrial partner, lab demonstrators, and students. We also present the results of our investigations to track the employment of our graduates over the last five years, identifying that all students who responded are employed within relevant positions or undertaking further study. Based on our successful delivery of the course, in this paper we make all tools and teaching materials available to the open source community, for delivery at other institutions. We emphasise the importance of the continued involvement of industry partners to inform pedagogical practices within Computer Science.