[Webinar] How to Protect Sensitive Data with CSFLE | Register Today

Presentation

Streampunk - The Difference Engine for Unlocking the Kafka Black Box

« Kafka Summit London 2022

"Kafka is like a black box, you can't see what's inside". How many times do we think and hear this? We all love Kafka but despite the plethora of existing tools, none of them have been able to break the seal of the "Kafka black box" just yet.

With Streampunk, I have set out to change this. Streampunk is a polyglot programmable shell for Kafka. It works quite like kcat or the Apache Kafka commandline tools, but unlike them, it is programmable using your favourite programming language instead of having to resort to the typical eccentric mix of Unix-shell scripting capabilities and tools. It is also a lot faster because Streampunk commands do not know the concept of startup time.

This lightning talk starts with a demo of how you can conveniently fulfill basic tasks such as listing and consuming topics using Streampunk in the Python interpreter. After that, I'll lead you through real-life examples of increasingly difficult challenges, such as copying a portion of a set of topics from one cluster to another, including a single message transform, and I'll show you how easily and elegantly these challenges can be solved with Streampunk.

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how