Developing Kafka Streams Applications with Upgradability in Mind

Does your organization struggle with updating of its Kafka Streams application? Releasing a new version of a Kafka Streams application can be challenging, especially if its state has to be preserved between releases. Consider these best-practices and architectural ideas to make this process smoother and improve your release process.

Having experienced accidental removal of change-log topics and needing to expand partitions, it is much easier to handle with some planning. With the proper planning, you can achieve easier application upgrades.

Key take-aways from the session include:

• How do minimize the rebuilding of the state-stores.
• How to change stream topologies without affecting the existing state stores.
• What you can do when you absolutely need to increase the number of partitions within your application.
• How to leveraging schemas for application releases.
• Measures to prevent data corruption, especially if Kafka is not only your system of record but also your source of truth.
• Techniques to support rolling back an application.
• The advantages of splitting apart a Kafka Streams application into multiple applications.