Level Up Your Kafka Skills in Just 5 Days | Join Season of Streaming On-Demand
"Kafka is like a black box, you can't see what's inside". How many times do we think and hear this? We all love Kafka but despite the plethora of existing tools, none of them have been able to break the seal of the "Kafka black box" just yet.
With Streampunk, I have set out to change this. Streampunk is a polyglot programmable shell for Kafka. It works quite like kcat or the Apache Kafka commandline tools, but unlike them, it is programmable using your favourite programming language instead of having to resort to the typical eccentric mix of Unix-shell scripting capabilities and tools. It is also a lot faster because Streampunk commands do not know the concept of startup time.
This lightning talk starts with a demo of how you can conveniently fulfill basic tasks such as listing and consuming topics using Streampunk in the Python interpreter. After that, I'll lead you through real-life examples of increasingly difficult challenges, such as copying a portion of a set of topics from one cluster to another, including a single message transform, and I'll show you how easily and elegantly these challenges can be solved with Streampunk.