Know Your Topics – A Deep Dive on Topic IDs with KIP-516

« Kafka Summit London 2022

When Apache Kafka® was first created, topics were identified solely by topic name—but this isn't always sufficient. Find out in this talk why the Kafka community decided to add topic IDs to Kafka as a part of KIP-516. Learn which new features related to topic IDs have been rolled out, and learn about some of the benefits that are still on the way.

We'll be covering new features in Kafka versions 2.8, 3.0, and 3.1 and how to upgrade to using topic IDs. We'll see how topic IDs are used in KRaft mode and tiered storage, and take a tour through some of the internals and the thought processes around these changes—as well as some of the future plans for topic IDs.

Presenter

Justine Olshan

Confluent

Justine graduated from Carnegie Mellon University in 2020 with a degree in computer science. During summer 2019, she was a software engineering intern at Confluent where she worked on improving the Apache Kafka® producer. After graduating, she returned full time to Confluent and continues to work on improving Kafka through various KIPs including KIP-516 which introduced topic IDs to Kafka and KIP-890 which strengthened the transactional protocol. She became an Apache Kafka committer in 2022 and PMC member in 2023.

Know Your Topics – A Deep Dive on Topic IDs with KIP-516

Presenter

Justine Olshan

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how