Getting Under the Hood of Kafka Streams: Optimizing Storage Engines to Tune Up Performance

With a few tweaks under the hood of your Kafka Streams implementation, you greatly improve performance. Sound too good to be true? Well, the secret lies in understanding storage engines.

You may already know that if you're using Kafka streams, you already have a storage engine in place, but do you know what options are available to tune it for optimal performance and scalability?

This presentation will discuss the importance of optimizing and choosing storage engines for Kafka streams applications.

Outline:

• What a storage engine is and how it relates to Kafka stateful streams
• The importance of understanding storage engines for optimal performance and scalability
• Evaluation of Storage Engines - Overview of popular storage engines, including Leveldb, Rocksdb, and Speedb open-source
• Review of the 5 most relevant configurable items and how they affect performance
• Practical ways to optimize and fine tune your storage engine
• Showcase - 2 minutes drop-in replacement demonstration