A Tale of Two Data Centers: Kafka Streams Resiliency

Running applications across two data centers is a requirement for many industries. Understanding how to deploy and architect a Kafka Streams application for multiple data centers can seem daunting for both developers and operators. Both stretch clusters and replication present unique challenges. This talk will go over best practices and answer questions such as, should I replicate internal topics? What are the implications of exactly once semantics? Do I need to run active/active or active/passive? How do I minimize recovery time after a failure? We’ll discuss important issues for stretch clusters such as rack/dc placement of internal topic partitions, state store gotchas and common latency vs throughput trade offs. The patterns presented will enable you to confidently design and execute resilient Kafka Streams applications.