The Best of Kafka Summit London 2023

Kafka Summit London 2023 has come and gone like any other year… But now, with all 60+ of the Kafka Summit London 2023 sessions available online, you can relive all of your favorite talks again and again!

Whether you were lucky enough to be a part of the action in-person in London this year, or you’re just now getting a chance to view the talks, you might be asking yourself where to start. Don’t worry—I’ve got you covered.

Using the audience ratings that we collected throughout the conference, I’m going to walk you through the top 10 sessions from Kafka Summit London 2023 so you know exactly which content to focus on first. 

Buckle up!

The top-rated sessions

10. A tie

We’re coming in hot with our 10th-place slot—and a tie! 

10. a. A 3-star microservices guide

First up, Olivier Jauze and Valérie Servaire of Michelin presented their session “From a monolithic orchestrator to streaming with micro-services.” The presentation outlined their experience going through a very common scenario in the streaming space: breaking down an existing monolithic application into microservices. But more than that, their implementation showcases an impressive use of Kafka Streams to help coordinate the over 150 microservices that came out of that breakdown.

It’s definitely worth a watch if your company is looking to tackle a looming monolith and bring some streaming into the mix. You can find the entire session here.

10. b. 🔥 Content

Also tied for 10th place is “Unveiling the Inner Workings of Apache Kafka® with Flamegraphs” from Divij Vaidya and Christo Lolov, two engineers representing AWS Cloud. Flamegraphs are incredibly useful tools that visually show how long certain functions are taking and ultimately allow you to see the performance of your applications. This session uses these visuals to dive into Kafka internals.

Whether you’re looking to understand how to use these tools to identify bottlenecks in your own clients or you’re generally curious to learn more about how to use flamegraphs, this is a great session to watch. Check it out here.

9. A lack of cheese sticks

The 9th highest-rated session came from Anna McDonald of Confluent, with her talk “Pragmatic Patterns (and Pitfalls) for Event Streaming in Brownfield Environments.” In this session, Anna introduces a number of practical patterns for tackling the realities that so many of us are faced with when tasked with bringing streaming into our existing tech stacks.

Anna has extensive experience in the field working with legacy systems, so this talk is a culmination of years’ worth of advice. It’s well worth the watch if you’re looking to integrate Kafka Streams. Give it a watch.

8. Another tie!

In 8th place, we have another tie… and I’ll let you in on a secret, it’s not the last tie we’ll see in the lineup.

8. a. K-Cuddle 🧸

First, a lightning talk from Decodable. Gunnar Morling presented an engaging session “Taming Kafka Connect with kcctl.” Kafka Connect is already a tricky technology to master… add to that the hassle of typos, long URLs, and curl, and it only gets worse! Gunnar’s talk introduces kcctl: a helpful command-line tool for managing and interacting with your Kafka Connect instances. In this short session, you’ll see a live demo of the basics of the tool and walk away with knowledge of some advanced features, as well.

If you’re looking for an easier way to manage Kafka Connect, this session is definitely for you. Check it out!

8. b.  Finserv, FTW!

Tied for the 8th place spot, we brought out the big guns. Kamlesh Shah from Morgan Stanley presented his session “Powering consistent, high-throughput, real-time distributed calculation engines using Kafka Streams.” In the talk, Kamlesh dives into a high-volume financial services application for liquidity management; his microservices approach makes heavy use of Kafka Streams.

Many special considerations go into streaming Finserv applications, so this session covers a ton of ground and is sure to appeal to a broad audience. If you’re curious to see how far you can push Kafka Streams in a high-throughput, low-latency environment, this is a great session to begin with. Watch it here.

7. ☁️ For beginners

Our 7th place spot for the best of Kafka Summit this year is taken by Steffen Hausmann from Materialize presenting “A Beginner’s Guide to Kafka Performance in Cloud Environments.” These days, if you’re looking to get started quickly with Kafka, you’ll likely start with a managed Kafka service in the cloud. Even though deployments are certainly easier in the cloud, it’s also much easier to accidentally rack up costs. Steffen is here to introduce you to a number of performance considerations (and the associated Kafka configurations) to keep in mind as you begin your Kafka journey in the cloud! By the end, you’ll have a better idea of how to optimize your cloud infrastructure moving forward.

If your goal is to keep your cloud-based Kafka costs in check, then this session is for you. Check it out here.

6. Surprise, a tie!

Y'all were really torn between the awesome Kafka Summit content this year! Another tie for the 6th place spot.

6. a.  ☠️☠️☠️

First up, Andrey Serebryanskiy from Raiffeisen Bank with “What to do if your Kafka Streams app gets OOMKilled?” If you’re deploying your streams applications on Kubernetes, you likely know just how frustrating an OOMKilled error can be. In this lightning talk, Andrey walks through how to debug this pesky error by diving into memory usage and strategies for monitoring and configuring at both the application- and pod-level.

Check out this session to be prepared the next time your pod crashes. Give it a watch here.

6. b.  A streams tongue twister

Tied for the 6th best talk at Kafka Summit London this year is Bruno Cadonna and Lucas Brutschy from Confluent presenting their session “Restoring restoration's reputation in Kafka Streams.” That’s a mouthful, but it doesn’t make the content any less useful! Restoring state in Kafka Streams is often viewed as a double-edged sword; on the one hand, it’s incredibly useful for fail-over and minimizing application downtime in the event that an instance goes down, but on the other hand, it can… take a while. Longer than we want in some cases. In this session, Bruno and Lucas explore tweaks that can help to streamline this restoration process and make it less “operationally problematic.”

Whether you’re a seasoned streams user or just looking to get started with the nuances of the technology, this will have something for you! Check it out here.

5. Some migration patterns

We’ve made it to the top 5, and we’re getting into the really good stuff now! Next, we have Jonathan Lew and Bhavesh Sooka from Synthesis Software Technologies with their session “The Journey of Migrating your System to the different Kafka Platforms available.” Need I say more? This is guaranteed to be a useful session for anyone who’s currently using Kafka and might consider migrating from one Kafka platform to another. There are so many different options and tradeoffs involved when first setting up your Kafka instances, and they’re just as important when migrating your instances. Covering networking, data access, client configurations, and more, Bhavesh and Jonathan walk through the different considerations you need to keep in mind when preparing to migrate your Kafka setup.

And remember, this talk isn’t just for on-prem operators or architects. It’s great knowledge for anyone. Give it a watch.

4. Did you think we were over with ties?

Really? It just means more great content for you to watch!

4. a.  Custom stores

Our first 4th place session is something that comes up a lot in Kafka Streams discussions. Daan Gerits walks us through “The Possibilities And Pitfalls of Writing Your Own State Stores.” Out-of-the-box, Kafka Streams is a beautifully flexible technology with a rich API of processors, but you can take it so much further with custom processors and… custom state stores! While opening the door to more, this custom functionality can potentially come at a cost. Know what to expect with Daan’s session where he explores adding searchability to his state stores.

If you’re thinking of implementing a custom state store for your next Kafka Streams application, or if you’re curious to hear more, check it out here.

4. b.  K8s clients

Another session on Kubernetes, but this time from the Kafka Consumer perspective. In his talk, “Intelligent, automatic restarts for unhealthy Kafka Consumers on Kubernetes,” Chris Shepherd of Cloudflare covers strategies for more proactively handling client pod failures with the goal of reducing the need for manual intervention.

If you’re considering using Kubernetes for your client deployments, seeing Chris’ team’s strategy might be helpful. Give it a watch here.

3. Fun and games

Congratulations! You’ve made it to the top 3 sessions in our best of Kafka Summit countdown! As a reward, we have a fun session for you. Dale Lane at IBM presented “Playing with Xbox data.” In it, he introduces Kafka Connect and Kafka Streams through a fun example using Xbox social and activity data.

If you like more tangible examples when learning technologies, this session is a great way to dive into Kafka. Check it out.

2. state_store_v2_final

Yet another Kafka Streams session! The 2nd highest-rated session is from Victoria Xia of Confluent who presented “Versioned State Stores in Kafka Streams.” Victoria’s session focuses on KIP-889 and the new functionality it brings to stream processing. Now users have the ability to add a time-based versioning component to their state stores and leverage temporal joins. The session outlines the basics of the new functionality with a stream-table join example as well as the motivation behind the KIP and parts of its implementation.

Versioned state stores are an awesome addition to Kafka Streams, and it's well worth knowing if you’re building streaming applications. Watch it here. 

1. To the Lonely Mountain

And finally, the top-rated session from Kafka Summit London is… 🥁

… in an unexpected and incredibly humbling turn of events, it’s “A Kafka Client’s Request: Or, There and Back Again,” a session that I presented! This session walks through the long, epic journey that a Kafka Producer request goes through to write data to the Kafka Broker. Starting from the producer side of things, you’ll see the different stages that the data goes through before being packaged into a producer request and sent on its way. Along the way, we cover a number of configurations that affect the producer request and the metrics you can use to monitor every stage.

Over the years, I’ve heard so many folks ask questions about producer metrics and configurations, so I’m happy to finally have outlined this content in a talk. Check it out.

Maintain momentum

That was our “best of” lineup—the top 10 rated sessions (even though there were actually 14 listed out with all of those ties!)—but I assure you that they weren’t the only great sessions this year. Kafka Summit London 2023 was jam-packed with incredible talks; so when you’re done catching up on the top-rated talks, give the rest of our speakers some love and check out their sessions, too.

And when you’re done… keep an eye out for the next round of streaming content at Current 2023 (and, of course, Kafka Summit London 2024).