Exactly-once Stream Processing Done Right

Exactly-once semantics is the holy grail in data stream processing, and Apache Kafka (including its stream processing library Kafka Streams) supports it. However, there is a lot of misunderstanding what exactly-once really is, what Kafka technically offers, where the limitations are, and how to use it correctly.

In this talk, we will dive into technical details to shed some light on the above questions. We approach the topic from a conceptual point of view, explain the challenges Kafka Connect faces when it comes to exactly-once, discuss how external source and sink systems can be integrated, and provide practical guidelines for implementing end-to-end exactly-once data pipelines correctly.