Kafka Connect is an ideal tool for building data pipelines. It is both reliable and scalable, with a pluggable interface that lets you flow data between Kafka and any system you need. A Connect pipeline is made up of many different components, and understanding how each of these interact together is essential, even for the simplest setup.
In this talk we will introduce the Connect components, from connectors, to transformations to the runtime itself. We will also share some of the new capabilities and best practices that you should be aware of to help you run and manage connectors effectively.
Finally we will talk about some different open source projects that have been built on top of Connect that can help you get the most out of the framework.
Presenter
Kate Stanley
Red HatKate Stanley is a Principal Software Engineer at Red Hat where she works on their Kafka offerings. She has experience running Apache Kafka on Kubernetes, developing enterprise Kafka applications and writing connectors for Kafka Connect. Alongside development, Kate has a passion for presenting and sharing knowledge. She is a regular speaker at technical conferences around the world and is also a Java champion. Kate started her journey with Kafka as part of the Event Streams team at IBM in 2018 and has worked on Kafka projects ever since.
Presenter
Mickael Maison
Red HatMickael Maison is a committer and the chair of the Project Management Committer (PMC) for Apache Kafka. He has been contributing to Apache Kafka and its wider ecosystem since 2015.
Mickael is a software engineer with over 10 years of software development experience. He is currently working in the Kafka team at Red Hat. He really enjoys sharing expertise and teaching and has been writing monthly Kafka digests since 2018 and enjoys presenting at conferences.