Building a Streaming Pipeline on Kubernetes Using Kafka Connect, KSQLDB & Apache Pinot

Managing Apache Kafka sometimes could be cumbersome, and that's something that we would like to avoid, especially for developers and data engineers that need to build and develop data pipelines.

Luckily, Kubernetes and Kafka's combination helps us reduce everyday tasks tremendously by adding myriad capabilities to lessen the complexity of managing clusters.

Kafka Connect and KSQLDB are a fantastic combo to add to your streaming stack. These two soldiers can facilitate data acquisition and processing and also provide outstanding real-time ETL capabilities. But what if you need an OLAP datastore to answer complex queries with a low-latency response, that's where Apache Pinot comes to play.

At this session, you're going to learn:

• Effective Kafka deployment on Kubernetes
• How to properly configure Kafka Connect and KSQLDB
• Integrate Apache Pinot to answer OLAP queries

Chinese Japanese Korean