OpenMessaging Benchmark: Measuring the Performance of Streaming Systems

While the database industry has worked on standardizing benchmarks for decades, with examples like the TPC consortium for relational databases and YCSB for the NoSQL world, there was really no such equivalent in the messaging and streaming system space.

This talk will present the OpenMessaging Benchmark, why it was created, and how one can use it to model messaging workloads and verify the behavior of different systems.

There are several aspects that characterize the performance and behavior of messaging systems: from the access to disk IO resources, to how efficiently the system can scale on multiple dimensions.

For users, it is often very difficult to gain a good understanding of all implications and impact that changes in the performance of the messaging system can have on their application.

We will also go through the set of properties that should be considered when comparing performance and unique challenges that are specific to messaging systems, along with an analysis of some of these systems like Apache Pulsar, Apache Kafka, RabbitMQ, and others.