Kafka Summit 2020: Day 1 Recap

Here is what happened on day one of the event—spoiler alert: My first Summit was awesome.

This year’s Kafka Summit is my first and I’ve been lucky to have a behind-the-scenes look at the event since joining Confluent in April. Our team and global community of speakers have been busy preparing, and it was exciting to finally see this year’s event get underway today. We had over 33,000 registrants help us kick off our first-ever virtual Kafka Summit!

Here are some of my Day 1 highlights in case you didn’t make it (don’t worry, all sessions were recorded and have been posted).

For a recap of the full event, check out this roundup by Tim Berglund.

Opening keynote with Gwen Shapira

Kafka Summit 2020 started with a keynote by Gwen Shapira, an engineering leader at Confluent and long-time Apache Kafka^® committer.

Gwen covered a number of technical and industry trends that have fueled Apache Kafka’s growth and also introduced future-ready patterns to give us a glimpse of where Kafka is heading next. This included covering some much-anticipated architectural changes to both the data and control planes of Apache Kafka, specifically:

• KIP-405 Kafka Tiered Storage: the ability to mix and match latency and throughput needs while maximizing elasticity and also storage options
• KIP-500 Replace ZooKeeper with a Self-Managed Metadata Quorum: Doubling down on shared log architecture to manage distributed state and dramatically simplify ops.

The open-source community has played an enormous role in helping shape these changes and I’m both grateful and excited to see how these changes will elevate how developers learn, use, and operate streaming platforms in the coming years.

Practical patterns from top Kafka users

Jun Rao, one of the original co-creators of Apache Kafka and one of the co-founders of Confluent, chatted with a number of notable Kafka users and Confluent customers.

As I watched the Q&A sessions, what struck me was how sophisticated and pervasive adoption was across industries. Whether it was a new digital-native (Nuuly), online travel (Expedia), global banking (Citigroup), or retail business (Lowe’s)—all these companies shared a similar journey from development to production.

Some common patterns that I observed across the various sessions with top companies:

• Starting small: Nearly every company’s Kafka origin story started with a small, non-mission critical use case. Many also used some sort of fully managed Kafka service like Confluent Cloud to get up and running quickly by abstracting away the ops burden of setting up and scaling Apache Kafka.
• Modernizing with microservices: Most organizations have been able to move from traditional messaging queuing systems to a real-time event streaming architecture as a by product of microservices adoption. Microservices have been both a precursor and accelerator for event streaming platform adoption in many use cases since Apache Kafka plays a key role in implementing patterns such as event sourcing and CQRS.
• Rise of the cloud-native data ecosystem: A perfect storm of technologies, providers, and skills has emerged to allow organizations to “sculpt their streams.” Whether aimed at high-scale transaction volumes, global distributed data systems, or industry-specific regulatory requirements—production-ready solutions are now within the grasp of most organizations to get the most out of Kafka, no matter their use case, infrastructure, or business requirements.

Other session highlights

Other day 1 sessions that stood out included a fascinating talk by Ben Stopford and Michael Noll from Confluent’s Office of the CTO. Taking us on a deep dive into the history and evolution of messaging systems, Ben and Michael covered the underlying principles and trade-offs between Apache Kafka, distributed log systems (Apache BookKeeper™ and its derivatives), and also Kafka-like messaging hubs (see performance benchmarks between popular options here). It’s a can’t-miss talk, especially if you’re an architect!

Other notable talks from Monday:

• Can Kafka Handle a Lyft Ride? (Andrey Falko & Can Cecen, Lyft)
• Building Event Streaming Applications with Pac-Man (Ricardo Ferreira, Confluent)
• Getting Started with Apache Kafka – A Contributor’s Journey (Israel Ekpo, Microsoft & Nikolay Izhikov, Sberbank & Matthias J. Sax, Confluent)

Taking a break

Between keynotes and sessions, I was also able to check out some fun parts of this year’s Kafka Summit, including various Birds-of-A-Feather gatherings, how to fly a drone, the official remix soundtrack, and even tips for yoga and at-home fitness.

And that was just Day 1!

Get ready for day 2

Hope you all get some rest and join us tomorrow morning as we start Day 2 with keynotes from Apache Kafka co-creator and Confluent CEO Jay Kreps, as well as industry luminary Sam Newman.

For the best of Kafka Summit 2020, check out this roundup by Tim Berglund.