Stream processing systems (SPSs) provide processing guarantees to ensure reliability under failure. However, no related work exists that empirically validates these guarantees. In this paper, we present PGVal, a tool that can end-to-end validate guarantees of SPSs. Additionally, we introduce new metrics for SPSs, such as reliability, reliable throughput, and failure cost, in addition to a refined definition of latency that results in improved measurements. We benchmark three popular SPSs, namely Kafka Streams, Apache Storm , and Apache Flink. Our results show that the reliability of SPSs depends on many characteristics, such as data rate, data partitions, processing topology, and parallelism factor. An SPS configuration may not continue to provide reliable outputs when any of these characteristics vary. PGVal can also inject faults into SPSs to observe their impact on reliability and performance. We provide a comprehensive failure model for fault-tolerance benchmarking of SPSs and report on the impact of faults on the reliability and performance of SPSs. Our experiments show that SPSs' reliability and performance drop varies by fault. Lastly, we provide suggestions to increase the reliability and performance of these systems.

Data usage is a significant concern, particularly in smartphone applications, M2M communications and for Internet of Things (IoT) applications. Messages in these domains are often exchanged with a backend infrastructure using publish/subscribe (pub/sub). Shared dictionary compression has been shown to reduce data usage in pub/sub networks beyond that obtained using well-known techniques, such as DEFLATE, gzip and delta encoding, but such compression requires manual configuration, which increases the operational complexity.

Publish/Subscribe is known as a scalable and efficient data dissemination mechanism. In a mobile environment, there is an added challenge for the pub/sub system to economizemobile bandwidth, which is especially precious in areas not wellcovered by mobile providers. While well-known compressionmethods such as GZip or Deflate are generally useful in suchsituations, we propose using Shared Dictionary Compression(SDC) to achieve a greater level of bandwidth efficiency. SDCrequires a dictionary, generated upfront, to be shared betweentwo communicating peers before it can be used. We proposea design where brokers forming the pub/sub overlay can be incharge of generating and propagating the shared dictionary. Oursolution employs an adaptive algorithm, executed at the brokers, which creates and maintains the dictionaries over time. Withthis approach, it is possible to reduce the required bandwidth byup to 88% including the introduced dictionary overhead. Ourdemo shows this approach applied to a smartphone applicationcommunicating with a publish/subscribe broker using the MQTTprotocol.